EE Seminar: 3D Multi-Modal FCNN for Automatic Segmentation of Anatomical ROIs for Seeding Pre-surgical Brain Tractography
Speaker: Itzik Avital
M.Sc. student under the supervision of Prof. Nahum Kiryati and Dr. Arnaldo Mayer
Wednesday, December 27th 2017 at 15:30
Room 011, Kitot Bldg., Faculty of Engineering
3D Multi-Modal FCNN for Automatic Segmentation of Anatomical ROIs for Seeding Pre-surgical Brain Tractography
Abstract
White matter tractography is a 3D modeling of neural tracts in the brain. It is an important tool for neuro-surgical planning and navigation. Stringent pre-operative time-constraints and limited availability of neuroanatomy experts indicate that pre-surgical tractography would clearly benefit from automation tools. Also, multi-modal methods, which use a number of different type inputs, have recently been found to be beneficial for many computer vision tasks. In this work, we propose a deep learning based method for automatic segmentation of seeding region-of-interest (SROI) for tractography. Moreover, we show that using the information from the white matter orientation patterns (PDD, Principal Diffusion Direction) achieves better results than using only the anatomical information (MRI scan). Also, we propose a novel approach for exploiting information from a number of inputs. This approach can be implemented by any encoder-decoder architecture. In this work, the motor, optic and arcuate tracts (right and left) are considered.
We defined five 3D encoder-decoder FCNNs (Fully Convolutional Neural Network). The first is fed with an MRI scan. The second is fed with a PDD map. The third is fed with an MRI scan and a PDD map which are concatenated as one input volume with 4 channels (early fusion). The fourth is composed of two streams, which are fused right before the two last layers; one is fed with an MRI scan and the other with a PDD map (late fusion). The fifth is an implementation of our novel approach. It is composed of two encoders (one for each modality) and one shared decoder (intermediate fusion). Each architecture has been trained for each tract type (arcuate, motor and visual) and over 5 cross-validation folds. A database consisting 75 cases, referred to Sheba Medical Center for brain tumor removal surgery, was used for the experiments. The presence of a brain tumor is clearly apparent in most of them which makes the task much more complex.
The superiority of the proposed architecture and the necessity of the PDD information for SROIs segmentation are confirmed by a number of metrics. Fully automatic tractography has been obtained using the automatic segmented SROIs and has been successfully validated against manual tractography. The proposed method provides a promising approach for automatic SROIs segmentation for tractography and can be used as an automatic tool.
