SRDTI: Deep learning-based super-resolution for diffusion tensor MRI
Qiyuan Tian, Ziyu Li, Qiuyun Fan, Chanon Ngamsombat, Yuxin Hu, Congyu Liao, Fuyixue Wang, Kawin Setsompop, Jonathan R. Polimeni, Berkin Bilgic, Susie Y. Huang
11 SRDTI: Deep learning-based super-resolution for diffusion tensor MRI Qiyuan Tian , Ziyu Li , Qiuyun Fan , Chanon Ngamsombat , Yuxin Hu , Congyu Liao , Fuyixue Wang , Kawin Setsompop , Jonathan R. Polimeni , Berkin Bilgic , Susie Y. Huang Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, United States; Harvard Medical School, Boston, MA, United States; Department of Biomedical Engineering, Tsinghua University, Beijing, China; Department of Electrical Engineering, Stanford University, Stanford, CA, United States. * Correspondence to: Qiyuan Tian, Ph.D., Athinoula A. Martinos Center for Biomedical Imaging, 149 13th Street, Charlestown, MA, 02129, United States. E-mail: [email protected].
Synopsis
High-resolution diffusion tensor imaging (DTI) is beneficial for probing tissue microstructure in fine neuroanatomical structures, but long scan times and limited signal-to-noise ratio pose significant barriers to acquiring DTI at sub-millimeter resolution. To address this challenge, we propose a deep learning-based super-resolution method entitled “SRDTI” to synthesize high-resolution diffusion-weighted images (DWIs) from low-resolution DWIs. SRDTI employs a deep convolutional neural network (CNN), residual learning and multi-contrast imaging, and generates high-quality results with rich textural details and microstructural information, which are more similar to high-resolution ground truth than those from trilinear and cubic spline interpolation.
Introduction
Diffusion tensor imaging (DTI) is widely used for mapping major white matter tracts and probing tissue microstructure in the brain . High-resolution DTI (e.g., 1.25 mm isotropic adopted by the Human Connectome Project (HCP) WU-Minn-Oxford Consortium ), if it were widely available, would be beneficial for probing tissue microstructure in fine neuroanatomical structures, such as cortical anisotropy and fiber orientations . However, long scan times and limited signal-to-noise ratio (SNR) pose significant barriers to acquiring DTI at high resolution in routine clinical and research applications. Higher resolutions would provide additional benefits , but it is challenging to acquire DTI data at millimeter and sub-millimeter isotropic resolution using standard 2D acquisitions with single-shot EPI readout, due to extremely high image blurring and distortion and low SNR. Slab acquisitions and multi-shot EPI have been employed for sub-millimeter DTI but require advanced sequences and reconstruction methods that are not widely available. Super-resolution imaging provides a viable way to achieve DTI at higher resolution in the spirit of image quality transfer . Previous studies have also demonstrated the feasibility of deep learning in super-resolution DTI . To address this challenge, we propose a deep learning method entitled “SRDTI” to synthesize high-resolution diffusion-weighted images (DWIs) from low-resolution DWIs. Unlike previous studies using a shallow convolutional neural network (CNN), SRDTI employed a very deep 3D CNN, residual learning and multi-contrast information sharing. We compared our results to those from image interpolation and quantified the improvement in terms of both image similarity and DTI quality. Methods
Data. Pre-processed T -weighted (0.7-mm isotropic) and diffusion data (1.25-mm isotropic, b=1,000 s/mm , 90 uniform directions) of 200 healthy subjects (144 for training, 36 for validation, 20 for evaluation) from the HCP WU-Minn-Oxford Consortium were used . Low-resolution data were simulated by down-sampling the high-resolution data to 2 mm iso. resolution using sinc interpolation. Co-registered T -weighted data were resampled to 1.25-mm isotropic resolution. Network Implementation. SRDTI utilizes a very deep 3D CNN (10 layers, 192 kernels per layer) to learn the mapping from the input low-resolution image volumes to the residuals between the input and output high-resolution image volumes (residual learning) (Fig. 1). The inputs of SRDTI are a low-resolution b=0 image volume and six corresponding DWI volumes up-sampled to 1.25-mm isotropic resolution, and a T -weighted volume. The outputs of SRDTI are a ground-truth high-resolution b=0 image volume and six corresponding DWI volumes. The input b=0 image volumes were obtained by averaging all acquired b=0 image volumes. The DWI volumes were synthesized from the fitted diffusion tensor sampled along six optimized diffusion-encoding directions, which minimize the condition number of the diffusion tensor transformation matrix , and were therefore equivalent to the six diffusion tensor components in the image space. Operating in the image space rather than the tensor component space improved data similarity in local regions and avoids unreliable tensor fitting in cerebrospinal fluid voxels. Anatomical images are often acquired along with diffusion data and were therefore included as an additional channel to outline different tissues and preserve Results
The b=0 image and DWIs at 1.25 mm isotropic resolution generated by SRDTI recovered more textural details and were visually similar to the ground-truth images (Fig. 2). The images were also quantitatively similar to the ground-truth images (Table 1a–c), with low MAEs around 0.012, high PSNRs around 31 dB and high SSIM around 0.98, which were 2 to 3 times better than those from trilinear and cubic spline interpolation. The residuals between the super-resolved images and ground-truth high-resolution images did not contain noticeable anatomical structure (Fig. 2, rows b, d, column iii). The V1-encoded FA maps from SRDTI displayed captured the striated appearance of the gray matter bridges spanning the internal capsule in the striatum (Fig. 3i), as well as known cortical anisotropy (Fig. 3ii) and the fine fiber pathways in the pons (Fig. 3iii). Quantitatively, the MAEs of the output DTI metrics were also low, with MAEs of 8.41°±0.35° for V1, 0.022±0.0009 for FA, and 0.029±0.002 μm /ms, 0.038±0.0019 μm /ms and 0.03±0.0019 μm /ms for MD, AD and RD, which were 35% to 80% of the MAE’s calculated for the corresponding DTI metrics obtained from trilinear and cubic spline interpolated images (Table 1d).
Discussion and Conclusion
We obtained high-quality super-resolution DWIs and DTI metrics at 1.25 mm isotropic resolution from low-resolution DWIs at 2 mm isotropic resolution (~4.1× voxel volume difference) using SRDTI, which employs a very deep 3D CNN and residual learning. Our results recover detailed microstructural information and demonstrate substantial improvement over the results derived from trilinear and cubic spline interpolation. SRDTI can be generalized to high-b-value data for mapping crossing fibers and more advanced microstructural models (e.g., diffusion kurtosis imaging and NODDI ). SRDTI can also be used to super-resolve sub-millimeter isotropic resolution images obtained from slab acquisitions and multi-shot EPI as well as lower-resolution images acquired using standard 2D sequences with single-shot EPI. Future work will compare SRDTI to other super-resolution methods. Acknowledgments
This work was supported by the NIH (grants P41-EB030006, U01-EB026996, R21-AG067562, K23-NS096056) and an MGH Claflin Distinguished Scholar Award.
References
1 Basser, P. J., Mattiello, J. & LeBihan, D. MR diffusion tensor spectroscopy and imaging.
Biophysical journal , 259-267 (1994). 2 Pierpaoli, C., Jezzard, P., Basser, P. J., Barnett, A. & Di Chiro, G. Diffusion tensor MR imaging of the human brain. Radiology , 637-648 (1996). 3 Sotiropoulos, S. N. et al.
Advances in diffusion MRI acquisition and processing in the Human Connectome Project.
NeuroImage , 125-143 (2013). 4 McNab, J. A. et al. Surface based analysis of diffusion orientation for identifying architectonic domains in the in vivo human cortex.
NeuroImage , 87-100 (2013). 5 Setsompop, K. et al. SLIce Dithered Enhanced Resolution Simultaneous MultiSlice (SLIDER-SMS) for high resolution (700 um) diffusion imaging of the human brain
In Proceedings of the 23rd Annual Meeting of ISMRM, Toronto, Ontario, Canada, 2015; Abstract 339 . 6 Setsompop, K. et al.
High ‐ resolution in vivo diffusion imaging of the human brain with generalized slice dithered enhanced resolution: Simultaneous multislice (g S lider ‐ SMS).
Magnetic resonance in medicine , 141-151 (2018). 7 Liao, C. et al. High-fidelity, high-isotropic-resolution diffusion imaging through gSlider acquisition with and T1 corrections and integrated ΔB0/Rx shim array.
Magnetic Resonance in Medicine , 56-67 (2020). 8 Wang, F. et al. In vivo human whole-brain Connectom diffusion MRI dataset at 760 μm isotropic resolution. bioRxiv , 2020.2010.2005.327395, doi:10.1101/2020.10.05.327395 (2020). 9 Wu, W., Koopmans, P. J., Andersson, J. L. & Miller, K. L. Diffusion Acceleration with Gaussian process Estimated Reconstruction (DAGER).
Magnetic resonance in medicine , 107-125 (2019). 10 Bilgic, B. et al. Highly accelerated multishot echo planar imaging through synergistic machine learning and joint reconstruction.
Magnetic Resonance in Medicine , 1343-1358, doi:10.1002/mrm.27813 (2019). 11 Hu, Y. et al. Motion ‐ robust reconstruction of multishot diffusion ‐ weighted images without phase estimation through locally low ‐ rank regularization. Magnetic resonance in medicine , 1181-1190 (2019). 12 Hu, Y. et al. Multi-shot diffusion-weighted MRI reconstruction with magnitude-based spatial-angular locally low-rank regularization (SPA-LLR).
Magnetic Resonance in Medicine , 1596-1607 (2020). 13 Alexander, D. C. et al. Image quality transfer and applications in diffusion MRI.
NeuroImage , 283-298 (2017). 14 Elsaid, N. M. & Wu, Y.-C. in et al.
The minimal preprocessing pipelines for the Human Connectome Project.
NeuroImage , 105-124 (2013). 16 Zhang, K., Zuo, W., Chen, Y., Meng, D. & Zhang, L. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing , 3142-3155 (2017). 17 Kim, J., Kwon Lee, J. & Mu Lee, K. Accurate image super-resolution using very deep convolutional networks. The IEEE Conference on Computer Vision and Pattern Recognition , 1646-1654 (2016). 18 Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition.
Preprint at: https://arxiv.org/abs/1409.1556 (2014).
19 Skare, S., Hedehus, M., Moseley, M. E. & Li, T.-Q. Condition number as a measure of noise performance of diffusion tensor data acquisition schemes with MRI.
Journal of Magnetic Resonance , 340-352 (2000). 20 Zhang, H., Schneider, T., Wheeler-Kingshott, C. A. & Alexander, D. C. NODDI: practical in vivo neurite orientation dispersion and density imaging of the human brain.
NeuroImage , 1000-1016 (2012). Figure 1. CNN architecture.
A very deep 3-dimensional plain convolutional neural network (CNN) comprised of stacked convolutional filters paired with ReLU activation functions ( n =10, k =192, d =3) is used for SRDTI. The CNN input is a b=0 image volume and six diffusion-weighted image (DWI) volumes along optimized diffusion-encoding directions interpolated to the target high resolution as well as an anatomical T -weighted image volume. The CNN output is the high-resolution b=0 image and six DWI volumes. Figure 2. Image results.
The b=0 images (row a) and diffusion-weighted images (DWIs) (row c) along one direction of the six optimized diffusion-encoding directions (i.e., [0.91, 0.416, 0]) from interpolated data (i, ii) (interpolated data using cubic spline is the input to SRDTI), SRDTI output data (iii), ground-truth high-resolution data (iv), and their residuals comparing to the ground-truth high-resolution images (rows b, d). A region-of-interest in the deep white matter (yellow boxes) is displayed in enlarged views (rows, b, d, column iv). Figure 3. Direction-encoded fractional anisotropy maps.
Fractional anisotropy maps color encoded by the primary eigenvector (red: left–right; green: anterior–posterior; blue: superior–inferior) derived from the diffusion tensors fitted using interpolated data (a, b), SRDTI super-resolution data (c), and ground-truth high-resolution data (d) along axial, coronal and sagittal directions, showing regions-of-interest in the internal capsule (i, ii), cerebral cortex (iii, iv) and pons (v, vi), respectively. Figure 4. Maps of DTI metrics.
Direction-encoded primary eigenvector (a), fractional anisotropy (c) and mean diffusivity (e) maps from interpolated data (i, ii), SRDTI super-resolution data (iii), ground-truth high-resolution data (iv), and their residuals comparing to the ground-truth high-resolution results (rows b, d). A region-of-interest in the deep white matter (yellow boxes) is displayed in enlarged views (rows, b, d, column iv). Table 1. Similarity quantification.