Spatially Regularized Parametric Map Reconstruction for Fast Magnetic Resonance Fingerprinting
Fabian Balsiger, Alain Jungo, Olivier Scheidegger, Pierre G. Carlier, Mauricio Reyes, Benjamin Marty
SSpatially Regularized Parametric Map Reconstructionfor Fast Magnetic Resonance Fingerprinting (cid:73)
Fabian Balsiger a,b,c,d, ∗ , Alain Jungo a,b , Olivier Scheidegger e,f , Pierre G.Carlier c,d , Mauricio Reyes a,b,1 , Benjamin Marty c,d,1 a ARTORG Center for Biomedical Engineering Research, University of Bern, Bern,Switzerland b Insel Data Science Center, Inselspital, Bern University Hospital, University of Bern,Bern, Switzerland c NMR Laboratory, Institute of Myology, Neuromuscular Investigation Center, Paris, France d NMR Laboratory, CEA, DRF, IBFJ, MIRCen, Paris, France e Department of Neurology, Inselspital, Bern University Hospital, University of Bern, Bern,Switzerland f Support Center for Advanced Neuroimaging (SCAN), Institute for Diagnostic andInterventional Neuroradiology, Inselspital, Bern University Hospital, University of Bern,Bern, Switzerland
Abstract
Magnetic resonance fingerprinting (MRF) provides a unique concept for simul-taneous and fast acquisition of multiple quantitative MR parameters. Despiteacquisition efficiency, adoption of MRF into the clinics is hindered by its dictio-nary matching-based reconstruction, which is computationally demanding andlacks scalability. Here, we propose a convolutional neural network-based recon-struction, which enables both accurate and fast reconstruction of parametricmaps, and is adaptable based on the needs of spatial regularization and thecapacity for the reconstruction. We evaluated the method using MRF T1-FF,an MRF sequence for T1 relaxation time of water (T1
H2O ) and fat fraction(FF) mapping. We demonstrate the method’s performance on a highly hetero-geneous dataset consisting of 164 patients with various neuromuscular diseasesimaged at thighs and legs. We empirically show the benefit of incorporating spa-tial regularization during the reconstruction and demonstrate that the methodlearns meaningful features from MR physics perspective. Further, we investigatethe ability of the method to handle highly heterogeneous morphometric varia-tions and its generalization to anatomical regions unseen during training. Theobtained results outperform the state-of-the-art in deep learning-based MRFreconstruction. The method achieved normalized root mean squared errors of0.048 ± H2O maps and 0.027 ± Keywords:
Magnetic resonance fingerprinting, convolutional neural network,quantitative magnetic resonance imaging, image reconstruction1 a r X i v : . [ ee ss . I V ] A ug . Introduction Magnetic resonance fingerprinting (Ma et al., 2013) (MRF) is a concept forsimultaneous and fast acquisition of multiple quantitative MR parameters. TheMR acquisition relies on temporal variations of MR sequence parameters usu-ally combined with high k -space under-sampling. As a result, a time-series ofweighted MR images is acquired, where each tissue has a unique MR signalevolution - or fingerprint. Such fingerprints can be simulated, e.g., by Blochequations, and a dictionary of expected fingerprints can be built. During im-age reconstruction, the acquired fingerprints are matched to this dictionary ofsimulated fingerprints with known MR parameters. The highest correlated fin-gerprint in the dictionary yields the MR parameters at the given voxel. Byrepeating this process for all voxels, parametric maps are reconstructed.MRF has the potential for clinically feasible multiparametric MR imaging,and could enable objective evaluation and comparison for a wide variety of clin-ical applications (Poorman et al., 2019). However, whilst the MRF acquisitionis fast, the dictionary matching reconstruction is computationally demandingand lacks scalability as the problem worsens exponentially with the number ofreconstructed MR parameters. For instance, as reported in Marty and Car-lier (2019a), the reconstruction of five parametric maps can require minutesto hours depending on the implementation and computational hardware. Thislong reconstruction is mainly attributed to the large dictionary with approxi-mately 9 million simulated fingerprints for five parametric maps. The recon-struction time will especially proof problematic when acquiring large data setswith many slices in clinically settings. Additionally, the dictionary matchingresults in discretized parametric maps, which might be undesirable consideringcontinuous-valued parametric maps using gold-standard MR sequences. There-fore, the dictionary matching represents currently a drawback of MRF, whichmakes a routine clinical application of MRF potentially inappropriate, and callsfor accurate and fast reconstruction alternatives.Several methods attempting to improve the MRF reconstruction have beenproposed lately. Acceleration of the dictionary matching (McGivney et al., 2014;Cauley et al., 2015; G´omez et al., 2016), iterative reconstruction (Davies et al.,2014; Pierre et al., 2016), and low-rank approximations (Mazor et al., 2018; (cid:73) F. Balsiger and M. Reyes designed, developed, and evaluated the deep learning compo-nent of the proposed solution. B. Marty designed the MRF T1-FF sequence and developedthe dictionary matching reconstruction pipeline. B. Marty and P. G. Carlier provided thepatient dataset. All authors provided critical feedback and helped to shape the research andmanuscript. ∗ Corresponding author.
Email address: [email protected] (Fabian Balsiger) B. Marty and M. Reyes share senior authorship.
Preprint submitted to Medical Image Analysis August 11, 2020 ssl¨ander et al., 2018; Zhao et al., 2018; Lima da Cruz et al., 2019) were pro-posed for MRF reconstruction. While some of these methods are promising,both further acceleration of the reconstruction process and continuous valuesin the parametric maps, as opposed to the discretely sampled dictionary-basedreconstruction, are highly desired. Promising in these regards are deep learning-based methods, which offer near real-time reconstructions and produce paramet-ric maps with continuous values.Deep learning-based methods for MRF reconstruction are versatile but cancoarsely be classified into fingerprint-wise reconstruction and spatially regu-larizing reconstruction. Fingerprint-wise methods feed single fingerprints intoa neural network that regresses the MR parameters of interest. Such meth-ods can directly be trained with the entries of the dictionaries but also withthe fingerprints of acquired MRF data. Hoppe et al. (2017) proposed a neu-ral network with three 1-D convolutions followed by a fully-connected layer forfingerprint-wise regression of MR parameters. Similarly, Cohen et al. (2018) re-lied on a solely fully-connected architecture with two hidden layers. A very sim-ilar fully-connected architecture was proposed by Golbabaee et al. (2019) withthree hidden layers. A more complex architecture was proposed by Song et al.(2019) using residual learning combined with attention mechanisms. Also, in thecontext of fingerprint-wise reconstruction, Virtue et al. (2017) investigated thecomplex-valued nature of MRF data by using a complex-valued fully-connectedneural network. Recently, Oksuz et al. (2019) used recurrent neural networks(RNNs), where the inputs to the RNN were also fingerprints, followed by afully-connected layer that regressed the MR parameters. The hypothesis thatinformation between neighboring fingerprints, especially in highly undersampledMRF, could benefit the reconstruction lead researchers exploring spatially regu-larizing methods. Here, a neighborhood of fingerprints is fed to a neural networkthat regresses a spatial patch in the parametric maps. These methods are usu-ally trained on acquired MRF data because spatial data, i.e., image slices, isrequired. Balsiger et al. (2018) proposed a convolutional neural network (CNN)regressing MR parameters from a neighborhood of 5 × ×
54 fingerprints are used for spatial regularization.Clearly, spatial regularization works superior to fingerprint-wise reconstruction,however, we argue that such strong spatial regularization, and especially spatialpooling and upsampling operations, as in Fang et al. (2018, 2019) is not needed.We hypothesize that the reconstruction performance is mainly dependent on1) the extent of spatial regularization and 2) the capacity of the CNN. Motivatedby the vast amount of MRF sequences and their differences in fingerprint di-mensionality (Poorman et al., 2019), we believe that a CNN architecture, whichis adaptable to the specific needs of different MRF sequences, is necessary. Toprove the hypothesis, we propose an algorithm that builds the CNN architecturebased on the needs of spatial regularization and capacity. The backbone of the3NN was presented in our conference contribution (Balsiger et al., 2019), whichwe extend by the algorithm making the CNN adaptable and possibly useful fordifferent types of MRF sequences. We evaluated the CNN’s performance ona large (n=164) and highly heterogenous patient dataset, and compared themethod to four existing deep learning-based methods proposed by Cohen et al.(2018); Hoppe et al. (2017); Oksuz et al. (2019); Fang et al. (2019). As inBalsiger et al. (2019), we empirically show the benefit of incorporating spatialregularization during the reconstruction and demonstrate that the CNN learnsmeaningful features from MR physics perspective. Additionally, we investigatedthe ability of the CNN to handle highly heterogeneous morphometric variationsand its generalization to anatomical regions unseen during training.
2. Materials and Methods
We used MRF T1-FF (Marty and Carlier, 2019a), an MRF sequence for T1relaxation time (T1) and fat fraction (FF) mapping in fatty infiltrated tissues.Fatty infiltration occurs for instance in neuromuscular diseases, where musclecells are irreversible replaced by fat resulting in loss of muscle strength. In suchcases, the FF in muscles is often quantified as biomarker for disease severity(Carlier et al., 2016; Paoletti et al., 2019). Additionally, increased T1 can befound in diseased muscle tissue (Marty and Carlier, 2019b), which might reflectdisease activity. However, in the presence of fatty infiltration, the T1 quantifi-cation can be biased by the fat, necessitating the separation of the water and fatpools resulting in T1 of water (T1
H2O ) and T1 of fat (T1 fat ). The MRF T1-FFsequence is specifically developed for such a separation of water and fat, and,is therefore capable of quantifying FF, T1
H2O , and T1 fat . The acquisition ofMRF T1-FF consisted of a 1400 radial spokes FLASH echo train following thegolden angle scheme after non-selective inversion. Echo time, repetition time,and nominal flip angle were varied during the echo train. The field of view wasset to 350 mm ×
350 mm with a voxel size of 1.0 mm × × fit scanner(Siemens Healthineers, Erlangen, Germany) using a set of 18-channel flexiblephase array coils, combined with a 48-channel spine coil.Five parametric maps were reconstructed after the MRF T1-FF acquisi-tion: FF, T1 H2O , T1 fat , and additionally the two confounding factors staticmagnetic field inhomogeneity (∆f) and flip angle efficacy (B1). First, the ac-quired data was transformed to image space using the non-uniform fast Fouriertransform (NUFFT) (Fessler and Sutton, 2003) with eight spokes per tem-poral frame, resulting in a highly undersampled time series of 175 temporalframes (acceleration factor of 68.7). Second, dictionary matching was con-ducted using a dictionary simulated by Bloch equations with (0:0 . , H2O , (225:25:400) ms for T1 fat , ( − . . S i g n a l i n t e n s i t y ( a . u . ) Temporal frame
WH T
ConvolutionalNeural Network FF T1 fat T1 H2O B1 Δf Figure 1: Schematic overview of the proposed MRF reconstruction. Patches of H × Wfingerprints with T temporal frames are extracted from MRF image slices and fed to a CNN,which simultaneously predicts all parametric maps. We used MRF T1-FF (Marty and Carlier,2019a), an MRF sequence to image diseased skeletal muscle. The parametric maps show thethighs of a 69 years old male patient with inclusion body myositis. FF: fat fraction, T1
H2O :T1 relaxation time of water, T1 fat : T1 relaxation time of fat, ∆f: static magnetic fieldinhomogeneity, B1: flip angle efficacy. the dictionary matching still required approximately 5 hours using standarddesktop computer hardware (2.6 GHz Intel Xenon E5-2630, 48 GB memory)due to the large number of MR parameter combinations. In summary, thetemporal dimensionality of the fingerprints of MRF T1-FF is 175, the spatialdimensionality is 350 × The hypothesis leading to the design of the proposed CNN architecture isthat the reconstruction performance depends on 1) the CNN’s receptive fieldand 2) the CNN’s capacity. On one hand, the receptive field determines thenumber of neighboring fingerprints the CNN will use to predict the value of theparametric maps of the central fingerprint. We argue that it is, especially in thecase of highly undersampled MRF, beneficial to leverage the spatial correlationamong fingerprints but this spatial correlation is limited to a certain extentdue to different tissue properties yielding different fingerprints, especially inlesions. Technically speaking, the extent of spatial correlation limits the numberof spatial convolutions, i.e., convolutions with kernel sizes larger or equal than3 ×
3. On the other hand, the reconstruction performance will also be determinedby the capacity of the CNN, i.e., the number of learnable parameters or numberof convolutional filter weights. A certain capacity is required to extract featuresthat cover the space of possible input fingerprints. Similar as for the receptivefield, an appropriate capacity is especially needed when dealing with multipleparametric maps and diverse tissue properties. Having both factors adaptableby an algorithm allows specific tailoring of the CNN-based reconstruction to theMRF sequence at hand.The schematic overview of the proposed MRF reconstruction is shown in5ig. 1. Let us consider a 2-D+time MRF image slice I ∈ C H × W × T after NUFFTin the image space with matrix size H × W and T temporal frames. We aim tofind the mapping M : I → Q from the MRF image space to M parametric maps Q ∈ R H × W × M . To learn this mapping, we use a 2-D CNN parametrized by itsconvolutional filter weights θ . The CNN processes the MRF data patch-wiseand treats the temporal frames as channels. Therefore, the CNN is trained tolearn the mapping f : I P → Q P , and estimates the parametric maps byˆ Q P = f ( I P ; θ ) , (1)where f the non-linear mapping of the CNN parametrized by θ , and I P ∈ C IP H × IP W × T ⊂ I and Q P ∈ R QP H × QP W × M ⊂ Q are patches extracted from the2-D+time MRF image slice and the parametric maps. The CNN reconstructsnon-overlapping patches of the parametric maps with size of QP H × QP W . Dueto the use of valid convolutions, the input patch size is larger and determinedwith the CNN’s receptive field R by IP H × IP W = QP H + R − × QP W + R − QP H × QP W = 32 ×
32 and the input patch dimension was IP H × IP W = 46 × ×
15, i.e., R = 15. Further for MRFT1-FF, H = W = 350, T = 175, and M = 5 (FF, T1 H2O , T1 fat , ∆f, B1).
The CNN architecture consists of temporal and spatial blocks, which areinterleaved within the architecture as shown in Fig. 2a. The temporal blocksextract temporal features from fingerprints while maintaining the receptive fieldof the CNN. The spatial blocks extract spatially correlated features and increasethe receptive field of the CNN. By appropriately setting the number of channelsin the temporal and spatial blocks, the capacity can be adjusted. The interleavedblocks are followed by a 1 × M channels for predicting Q P . Input to the CNN are real-valued I P with realand imaginary parts concatenated as 2 T channels. A temporal block (Fig. 2b)consists of 1 × L layers, andeach of them is a sequence of 1 × C T filters (growthrate), and the feature maps of the preceding layers are concatenated before thenext layer to reuse features and facilitate the gradient flow (Huang et al., 2017).A spatial block (Fig. 2c) extracts features from neighboring fingerprints, and,therefore, increases the CNN’s receptive field. It consists of a valid 3 × C S filters followed by ReLU activation function, and BN. All The patch-wise processing is mainly motivated by graphics processing unit (GPU) memorylimitations, in this study 12 GB. T x x
256 550 x x x
224 454 x x
44 42 x x x x
179 192 x x x M x x IN H x W C S H - x W - C I N H x W C I N + C T H x W H x W H x W C I N + C T C I N + C T temporal block spatial block conv 1 x 1, linear conv 3 x 3, ReLU, BNconv 1 x 1, ReLU, BNC T L concat(a) (b) (c)
Figure 2: The proposed CNN for MRF reconstruction with a receptive field of 15 ×
15 ( R =15). (a) the architecture for MRF T1-FF and its (b) temporal (here L = 3) and (c) spatialblocks. The bars indicate feature maps with the number of channels indicated on the topand the spatial size indicated on the lower left. T: number of temporal frames, M: number ofparametric maps, BN: batch normalization, ReLU: rectified linear unit, L: number of layersin a temporal block, C T / C S : number of channels in a temporal/spatial block, C IN : numberof input channels, H × W: feature map size. convolutions in the CNN are performed with a stride of 1. In principle, thetemporal blocks extract a high number of channels, from which the spatialblocks then extract spatially correlated features with an even higher numberof parameters (factor 9 due to 3 × We propose Algorithm 1 to parametrize the temporal and spatial blockssuch that the receptive field and the capacity of the CNN are as desired for theMRF sequence to reconstruct. Inputs to the algorithm are the receptive field R , the number of parameters N P , i.e., the number of learnable weights of allconvolutional kernels in the CNN, and the number of non-linearities N L , i.e.,the number of ReLU activation functions . The number N B = ( R − / C S are chosen such that it gradually decreasesdown to C S stop , in steps of C S dec , before the last convolutional filter with linearactivation, i.e., C S = (cid:0) iC S dec + C S stop (cid:1) i = N B − , (2)where ( · ) denotes a sequence (e.g., C S = (160 , , ,
64) for N B = 4, C S stop =64, and C S dec = 32). The number of layers L in each of the temporal blocks aredetermined by Note that the number of non-linearities are equal to the number of convolutions in thetemporal and spatial blocks because each convolution is directly followed by a ReLU activationfunction. = (cid:18)(cid:22) N L T N B (cid:23) + ( N LT mod N B ) ≤ i (cid:19) N B i =1 , (3)where N L T = N L − N B is the remaining number of non-linearities, which corre-sponds to the total number of convolutional layers in all temporal blocks. Theindicator function returns 1 if the statements is true and 0 otherwise, andit allows uneven distribution of the convolutional layers among the temporalblocks (e.g., L = (3 , , ,
2) for N B = 4 and N L T = 10). Given that the numberof filters of the temporal blocks C T gradually decrease down to C T stop , in stepsof C T dec , an ideal number of channels in the first temporal block C T start can becalculated such that the total number of learnable convolutional parameters areas close as possible to the desired number of learnable parameters N P . There-fore, the desired capacity of the CNN can be matched as close as possible. C T is calculated by C T = ( g ( i )) N B − i =0 , (4)with g ( x ) = (cid:40) C T start − xC T dec , if C T start − xC T dec ≥ C T stop C T stop , otherwise , (5)(e.g., C T = (80 , , ,
32) for N B = 4, C T start = 80, C T stop = 32, and C T dec =32). As we will analyze in Section 3.3, the optimal receptive field of the CNNfor the MRF T1-FF sequence is 15 ×
15 with approximately 5 million pa-rameters. Therefore, the architecture consists of seven temporal and spatialblocks. We empirically chose the hyperparameters N L = 21, C S stop = 64, C S dec = 32, C T stop = 32, and C T dec = 32, resulting in number of layers L =(2 , , , , , , , C T = (179 , , , , , , C T start = 179, and channels of the spatial blocks C S = (256 , , , , , , Calculating the number of parameters N of a convolution is straightforward, i.e., K ∗ C in ∗ C out + C out for a convolution with kernel size K × K , C in input channels, and C out output channels. lgorithm 1 Algorithm for CNN architecture building.
Input: R , N P , N L , C S stop , C S dec , C T stop , C T dec Output:
L, C T , C S N B ← ( R − / N L T ← N L − N B Calculate C S with Eq. 2 Calculate L with Eq. 3 C T start ← N P best ← ∞ loop Calculate C T with Eq. 4 N P current ← calculate parameters ( L, C T , C S ) if | N P − N P current | < N P best then N P best ← N P current C T start ← C T start + 1 else if | N P − N P current | > N P best then C T start ← C T start − Calculate C T with Eq. 4 return L, C T , C S else return L, C T , C S end if end loop The CNN was trained for 75 epochs with a batch size of 20 randomly se-lected patches, which we empirically found to be sufficient. The Adam opti-mizer (Kingma and Ba, 2015) was used to minimize a mean squared error (MSE)loss ( (cid:96) ) with a learning rate of 0 . β = 0 .
9, and β = 0 . θ (cid:88) i (cid:96) ( f ( I P i ; θ ) , Q P i ) . (6)Before training, we normalized the data subject-wise: The MRF image wasnormalized to zero mean and unit standard deviation along the real and imag-inary parts and each parametric map was rescaled to the range [0 ,
1] using theminimum and maximum values in the dictionary (see Section 2.1). After infer-ence, the predicted parametric maps were rescaled back to the original dictionaryrange. The CNN was implemented in TensorFlow 1.10.0 (Google Inc., MountainView, CA, U.S.) with Python 3.6.7 (Python Software Foundation, Wilmington,DA, U.S.). The training was performed with an NVIDIA TITAN Xp (NvidiaCorporation, Santa Clara, CA, U.S.). For reproducibility, the source code isavailable online . Further investigation of some architecture hyperparameterscan be found in Section 1 of the supplementary material. https://github.com/fabianbalsiger/mrf-reconstruction-media2020 able 1: Clinical and demographic information of the dataset and its distribution into training,validation, and testing splits. Values are given as mean age ± standard deviation and numberof total subjects / number of male subjects / number of thigh images. BMD: Becker musculardistropy, DMD: Duchenne muscular distropy, IBM: Inclusion body myositis. SplitDisease Overall Training Validation TestingBMD 45.4 ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± We evaluated the performance of the proposed method on a clinical datasetconsisting of 164 patients with various neuromuscular diseases (NMDs). Thedataset is highly heterogeneous due to the variable phenotypic appearance oflesions in NMDs, and further comprises thigh and leg images. To evaluatethe robustness of the methods, we purposely did not apply any stratificationregarding disease type, patient sex, patient age, or anatomical region whensplitting the dataset into training, validation, and testing splits (n=94/20/50).Table 1 summarizes clinical and demographic characteristics of the dataset.Multimedia files characterizing the heterogeneity of the dataset can be foundonline as supplementary material.The dictionary matching reconstruction served as reference for the para-metric maps. Quantitative analysis between the dictionary matching and thepredicted parametric maps was done according to Zbontar et al. (2018). Thenormalized root mean squared error (NRMSE), the peak signal-to-noise ratio(PSNR), and the structural similarity index measure (SSIM) (Wang et al., 2004)were calculated at the image level. Due to the quantitative nature of paramet-ric maps, we provide further quantitative analysis based on the coefficient ofdetermination (R ), scatter plots, and Bland-Altman analysis. To this end, wemanually segmented regions of interest (ROIs) lying within the major musclesof each subject. The ROIs allowed calculating the mean parametric value withineach ROI of each image slice. Then a linear regression between the mean ROIvalues of the dictionary matching and predicted parametric maps quantified theagreement between the methods. For all evaluation, background voxels (air)were excluded based on an automatically segmented mask generated by thresh-10lding an anatomical image obtained from the MRF image space series (pseudoout-of-phase image (Marty and Carlier, 2019a)). Further, voxels and ROIs witha FF higher than 0.7 were excluded from the evaluation of NRMSE, PSNR, andR of the T1 H2O map reconstruction due to low confidence of T1
H2O at highFF (Marty and Carlier, 2019a). For the SSIM, we used a window size of 7 × K = 0 . K = 0 .
03, and L was set to the maximum value of the parametricmap.We compared the proposed method to four other deep learning-based MRFreconstruction, which can be grouped into methods working fingerprint-wise anda method considering spatial neighborhoods of fingerprints. The fingerprint-wise methods comprise Cohen et al. (2018), a neural network with two hiddenfully-connected layers, Hoppe et al. (2017), a CNN with four 1-D convolutionallayers, and Oksuz et al. (2019), a recurrent neural network (RNN) using agated recurrent unit with 100 neurons. The spatial method proposed by Fanget al. (2019) works patch-wise using an U-Net-like CNN with pooling operationsresulting in a receptive field of 54 ×
54. We implemented all competing methodsas described in the papers due to lack of publicly available code. The inputand output dimensions were adapted for MRF T1-FF using the complex-valuedMRF data as input for all methods. For training, we used the Adam optimizerwith a MSE loss as for the proposed method. The batch sizes were set to 100 forthe fingerprint-wise methods and the feature extraction module of Fang et al.(2019), and to 20 for the spatially-constrained quantification module of Fanget al. (2019). Training was performed for 25 epochs (Cohen et al., 2018; Hoppeet al., 2017; Oksuz et al., 2019) and 75 epochs for Fang et al. (2019). For eachmethod, the learning rates were chosen from the set { . , . , . } basedon the performance on the validation set.
3. Experiments and Results
Reconstruction results of the dictionary matching, the proposed method,the best fingerprint-wise method (Oksuz et al., 2019), and the spatial methodof Fang et al. (2019) are shown in Fig. 3. Visually, the proposed method achievedthe best reconstruction results for all parametric maps. Compared to the dictio-nary matching, all reconstructions appear to be slightly smoothed. Oksuz et al.(2019) resulted in noisier reconstructions and could not capture elevated T1
H2O .Fang et al. (2019) achieved similar results as the proposed method, but the re-constructions contain artifacts, which are not present for the proposed method.The artifacts possibly originate from the patch-wise processing in combinationwith padding convolutions, i.e. equal spatial dimension of the input and out-put of their CNN, resulting in boundary effects. Further, the reconstructions ofFang et al. (2019) appear to be slightly more smooth than the reconstructionsof the proposed method. A zoomed-in region with fatty infiltrated muscle andelevated T1
H2O can be found in Section 2 of the supplementary material, show-ing that Oksuz et al. (2019) fails to reconstruct elevated T1
H2O and that thereconstructions of Fang et al. (2019) contain subtle reconstruction artifacts.11 ictionary matching Proposed Oksuz et al. Fang et al. FF T H O ( m s ) T f a t ( m s ) Δ f ( H z ) B ( a . u . ) Figure 3: Parametric map reconstruction results of a thigh of a 71 years old male patient withinclusion body myositis. Reconstructions of the dictionary matching, the proposed method,Oksuz et al. (2019), Fang et al. (2019), and the error (dictionary minus reconstruction) areshown. a.u.: arbitrary unit.
Quantitatively, the proposed method achieved the best reconstruction resultsfor the four metrics (Table 2). The parametric maps T1
H2O and T1 fat are themost difficult to reconstruct while for FF, ∆f, and B1 the quantitative results arebetter. The quality of the agreement is further shown by the quantitative analy-sis of the ROIs in Fig. 4. The correlations between the CNN and the dictionary12 able 2: Quantitative results of the proposed and the compared methods. The metrics nor-malized root mean squared error (NMRSE), peak signal-to-noise ratio (PSNR), structuralsimilarity index measure (SSIM), and coefficient of determination (R ) were calculated forthe five parametric maps. MethodMetric Parametricmap Proposed Hoppe et al. Cohen et al. Oksuz et al. Fang et al.NRMSE FF ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± matching reconstruction are very high with the Pearson correlation coefficient r > .
95 (Fig. 4, left column). Only for T1
H2O and T1 fat , the agreements areslightly decreased with R s of 0.919 and 0.927. The Bland-Altman plots (Fig. 4,right column) show small to no bias for all five parametric maps, and the 95 %limits of agreement are smaller than the dictionary sampling increment for FF,T1 fat , ∆f, and B1 (cf. Section 2.1). For T1 H2O , the agreement between themethods is approximately ± ±
60 ms. Similar plots for theOksuz et al. (2019) and Fang et al. (2019) can be found in Section 2 of the sup-plementary material. Reconstructing the parametric maps of one subject (fiveimage slices) required approximately 1 second with the proposed method, whichis considerably faster than the dictionary matching requiring up to minutes oreven hours depending on the implementation (McGivney et al., 2019). Our dic-tionary matching implementation requires approximately 5 hours per subject(Marty and Carlier, 2019a). The compared deep learning-based methods arealso in the range of 1 second with the fingerprint-wise methods (Hoppe et al.,2017; Cohen et al., 2018) being slightly slower than the proposed CNN, followedby the RNN of Oksuz et al. (2019). The two-stage process of Fang et al. (2019)resulted in the longest reconstruction times.
In a post hoc analysis, we investigated the blurriness (or smoothness) ofthe reconstructions of the different methods. We analyzed the energy of thehigh frequencies in the parametric maps as a metric of blurriness, i.e., the ratiobetween the energy of the high frequencies and the energy of all frequencies(similar to Section 2.1 of the supplementary material of Fang et al. (2019)).We defined the high frequencies in the spectrum of the parametric maps to be13 igure 4: Quantitative agreement between the proposed method and the dictionary matching.(left) Scatter and (right) Bland-Altman plots for the five parametric maps where each dotrepresents the mean value of the parametric map for a manually segmented ROI lying withina major muscle (n=4392 for FF, T1 fat , ∆f, and B1, and n=3943 for T1
H2O ). For the scatterplots, the solid line indicates x=y and the dashed line indicates the fit of the linear regression.For the Bland-Altman plots, the solid line indicates the mean difference and dashed linesindicate the 95 % limits of agreement between the dictionary matching and the proposedmethod. the frequencies above a certain threshold, which we varied from 55 to 95 %because defining one single threshold to separate low and high frequencies wasdifficult. The energy was defined as the sum of the squared magnitudes. Fig. 514 H i g h - f r e q u e n c y p r o p o r t i o n Dictionary matching Proposed Oksuz et al. Fang et al.
Figure 5: Blurriness of the T1
H2O map reconstruction. The bars indicate mean ± standarddeviation. compares the blurriness of the T1 H2O map reconstruction between the methods(see Section 3 of the supplementary material for the FF, T1 fat , ∆f, and B1maps). For T1
H2O , all methods clearly produced smoother reconstructions thanthe dictionary matching. Further, the visually smoother appearance of thereconstructions of Fang et al. (2019) can be confirmed quantitatively. For the FFand ∆f maps, Oksuz et al. (2019) resulted in less smoothing than the dictionarymatching, which also confirms the noisy appearance in Fig. 3.
The influence of the spatial dimension on the reconstruction quality, or inother words, to what extent the correlation of neighboring fingerprints is ben-eficial for the reconstruction, is to this date not well studied. Therefore, wevaried the receptive field of the proposed CNN from 1 ×
1, which correspondsto fingerprint-wise reconstruction, up to a receptive field of 21 ×
21 using Al-gorithm 1. We further varied the number of parameters from 1 to 5 million insteps of 1 million (see Section 4 of the supplementary material for configura-tions). Fig. 6 shows the R of the T1 H2O map reconstruction depending on thereceptive field and the number of parameters . It is visible that fingerprint-wisereconstruction results in significantly inferior reconstructions. Receptive fieldsaround 15 ×
15 seem to perform well with little to no added value when incor-porating more fingerprints for the reconstruction. There is no significant changein performance with fewer or more number of parameters. The influence on theparametric maps and metrics except for the T1
H2O map and R were less ac-centuated for receptive fields above 5 ×
5. Increasing the receptive field beyond15 ×
15 had in some cases negative influence (see Section 4 of the supplemen-tary material). Therefore, we did not experiment with receptive fields beyond This experiment led to the choice of the final architecture with a receptive field of 15 × igure 6: Influence of the spatial receptive field and the number of parameters on the T1 H2O map reconstruction. The numbers denote the R .Table 3: Architectural summary of the proposed method and the methods of comparison. ForFang et al. (2019) the numbers represent the sum of the numbers of the feature extractionand the spatially-constrained quantification module.Method Number of parameters Number of non-linearities Receptive fieldProposed 5.00 million 21 15 × × × × × ×
21. Considering all parametric maps and metrics, we chose a receptivefield of 15 ×
15 and 5 million parameters. For comparison, we summarize thereceptive field and the number of parameters of the methods of comparison inTable 3.
The influence of the temporal dimension, or in other words, to what extentthe temporal frames contribute to the reconstruction, might be of valuable in-formation for the MRF community. To investigate the influence of the temporaldimension, we reformulated the permutation importance by Breiman (2001a,b)for MRF. The permutation importance measures the importance of a variable toa model’s prediction accuracy when the variable is permuted Fisher et al. (2019).Here, the variables are the fingerprint intensities of each temporal frame. We,therefore, randomly permuted the intensities of the t -th temporal frame andreconstructed the parametric maps using this permuted MRF data as input.The absolute difference in NRMSE to the non-permuted reconstruction is thenconsidered as the importance of the t -th temporal frame of the MRF sequence.The importance of all temporal frames for reconstructing the five parametricmaps is shown alongside the MRF sequence in Fig. 7. The first few temporalframes after the non-selective inversion pulse, which should be sensitive to T1,have the highest importance for the reconstruction of the T1 H2O map. For T1 fat the temporal frames after 125 are the most important when water and fat areout of phase. Generally, the temporal frames after the changes in the MRF T1-FF sequence parameters result in high importance for the reconstruction (i.e.at temporal frames 75, 100, 125, and 150).16 able 4: Quantitative results for the reconstruction of unseen anatomical regions. The num-bers denote the R . Parametric mapMethod FF T1
H2O T1 fat ∆f B1Proposed Oksuz et al. (2019) 0.746 0.409 0.420 0.862 0.760Fang et al. (2019) 0.973 0.758 0.787 0.970 0.980
The results show that the proposed method reconstructs highly heteroge-neous morphometric variations in NMD patients well. However, it is unclearhow many training subjects are actually needed to obtain a model with goodrobustness. To investigate this, we randomly selected subsets of a varying num-ber of training subjects from the training set, trained the proposed method withthese subsets, and reconstructed the testing set to assess the robustness. Thewhole process was repeated five times. Fig. 8 summarizes the results of this ex-periment. With 40 training subjects, the proposed method reconstructs almostidentically as when using the entire training set with 94 subjects. However,we also observe that the number of required training subjects depends on themetric of interest.
The generalization of deep learning-based MRF reconstruction methods tounseen anatomical regions during training has not been investigated so far, tothe best of our knowledge. Therefore, we imaged three NMD patients at threeanatomical regions distinctly different to the thigh and leg: the shoulder, thelower abdomen, and the pelvis. MRF acquisition, reconstruction, and evalua-tion were identical as described in Section 2.1 and Section 2.6. FF and T1
H2O map reconstructions of the proposed method are shown in Fig. 9 and the R s ofthe ROI analysis for all parametric maps and methods are summarized in Ta-ble 4 (see Section 5 of the supplementary material for all metrics). Qualitatively,the proposed method reconstructed the parametric maps with good quality. Intissues other than skeletal muscle and fatty tissue, the dictionary matching andthe proposed reconstruction resulted in noisy parametric maps, which was ex-pected due to the MRF T1-FF sequence’s purpose. Quantitatively, the proposedapproach resulted in a slight decrease of performance for FF, ∆f, and B1. ForT1 H2O and T1 fat , the decrease is more significant (cf. Table 4). This decreasewas even more accentuated for the method of Fang et al. (2019). And, despiteworking fingerprint-wise, the method of Oksuz et al. (2019) resulted in the worstreconstructions for unseen anatomical regions.17 . Discussion
We have investigated the reconstruction of parametric maps from MRF us-ing CNNs. Driven by the hypothesis that the reconstruction performance de-pends on the incorporation of neighboring fingerprints, i.e., the receptive fieldof the CNN, and the capacity, i.e., the number of parameters of the CNN, wehave designed an algorithm for flexible architecture building based on the spe-cific requirements of the MRF sequence to reconstruct. The configuration forMRF T1-FF was empirically determined to be a receptive field of 15 ×
15 withfive million parameters. With this configuration, we have shown that the pro-posed method yields accurate parametric map reconstruction, independent ofthe morphometric heterogeneity of imaged patients as well as unseen anatom-ical regions. The method is fast, enabling reconstruction of parametric mapsin a clinical setting. Further, as shown qualitatively and quantitatively, betterreconstruction results were achieved with the proposed method as compared toother deep learning-based methods.The proposed method yielded an absolute reconstruction error lower thanthe dictionary sampling increment for all except the T1
H2O map (Fig. 4). There-fore, we argue that there is no difference in reconstruction accuracy between theproposed method and the dictionary matching for the FF, T1 fat , ∆f, and B1maps. Based on the observed differences for T1
H2O map reconstructions, weconcede that the sensitivity of MRF T1-FF to T1
H2O might not be optimal.The fingerprints encode T1
H2O to some extent, but for nuances in T1
H2O , theycontain probably more noise than discriminative patterns. This observationcan also be confirmed when comparing simulated fingerprints with close T1
H2O values (see Section 6 of the supplementary material). Further, large receptivefields were mainly beneficial for T1
H2O with a lower effect on the other para-metric maps. The CNN might compensate for the low signal-to-noise ratioby regularizing spatially. We, therefore, also believe that modifications of theCNN architecture will bring limited additional reconstruction performance andthat efforts are better invested at optimizing the MRF sequence than the deeplearning-based reconstruction such as done recently (Cohen and Rosen, 2017;Zhao et al., 2019; Lee et al., 2019).We have analyzed that the receptive field, i.e., considering a spatial neigh-borhood of fingerprints, influences the reconstruction performance. On the onehand, fingerprint-wise reconstruction (receptive field of 1 ×
1) is significantlyinferior to spatial reconstruction. We attribute this mainly to the potentiallyhigh correlation of neighboring fingerprints coupled with the strong undersam-pling of MRF. On the other hand, spatial reconstruction has improved thereconstruction only to some extent (Fig. 6). Regarding the blurriness of thereconstruction, the optimal receptive field is a difficult choice but it lies likelybetween fingerprint-wise reconstruction and the large receptive field of Fanget al. (2019) (Fig. 5). Further, we have observed a larger decrease in perfor-mance for Fang et al. (2019) when reconstructing unseen anatomical regions.We attribute this decrease to the method’s large receptive field of 54 ×
54 wherefingerprints without valuable information, possibly influenced by susceptibility18rtifacts, were included in the reconstruction (cf. noisy tissues in the para-metric maps). Therefore, we conclude that pooling operations with subsequentdeconvolution operations, as e.g. in the U-Net-like architecture of Fang et al.(2019), are not needed for MRF reconstruction. Clearly, spatial regularizationis superior to fingerprint-wise reconstruction but its extent dependents almostcertainly on the MRF sequence due to various factors such as the sensitivity tothe MR parameters, k -space sampling, in-plane voxel size, among others. Butwith the proposed algorithm, investigating this aspect becomes straightforwarddue to the CNN’s adaptability.We have studied the influence of the temporal frames on the parametricmap reconstruction. Such reconstruction interpretability might be useful forfurther developments of MRF reconstruction, as well as the MRF sequence de-velopment itself. As expected, the inversion pulse yields high importance tothe first few temporal frames for T1 parameters. The general correlation be-tween abrupt sequence parameter changes and high importance hints at highlysensitive temporal frames to MR parameters, and, therefore, rich informationfor the reconstruction. Fang et al. (2019) proposed to reduce the time of theMRF acquisition by considering only the first fractions of the temporal frames.However, such an approach, although being straightforward, might be subopti-mal considering that not all temporal frames might be of equal importance forthe MRF reconstruction. For instance, in the case of MRF T1-FF, the tem-poral frames 50 to 75 as well as the last 25 temporal frames might be uselessfor the reconstruction, containing maybe redundant or irrelevant information.An acceleration of the MRF acquisition might be achieved by discarding thesetemporal frames without sacrificing reconstruction performance.We have demonstrated an excellent robustness of the proposed method toheterogeneous morphometric variations and unseen anatomical regions, a de-sired key property for image reconstruction (Knoll et al., 2019). PreviousMRF reconstruction studies were performed on small cohorts of healthy volun-teers (Cohen et al., 2018; Balsiger et al., 2018; Fang et al., 2018, 2019; Golbabaeeet al., 2019; Song et al., 2019), limiting their clinical significance. Here, we havepresented, to the best of our knowledge, the first study on reconstructing highlyundersampled MRF of patient data only (Table 1). To study the robustness of amethod, NMDs are an excellent subject given their large phenotypic variability,the broad range of affected anatomical regions as well as patient age distribu-tion. Our approach seems to be rather insensitive to such variability, which wealso attribute to the large and heterogeneous training data. We have found thatit is also possible to achieve good robustness with fewer training data (Fig. 8).However, the robustness is dependent on the parametric map as well as thedesired property of the reconstruction (e.g., structural similarity and signal-to-noise). Further, we have found that the number of parameters of the CNN isa rather insensitive characteristic (Fig. 6). Nor a decrease in performance withfewer neither overfitting with more learnable parameters have been observed,indicating that a performance benefit can primarily be attributed to the spa-tial regularization (i.e., the receptive field). Reproducing the results of Fig. 8with varying number of parameters might give additional insights into a possible19ink between robustness and the number of parameters but is computationallyunfeasible due to the immense number of training runs needed.Our study has several limitations, which we plan to address in future work.First and foremost, the absence of a better reference for comparison than the dic-tionary matching is a significant issue. Ideally, the CNN reconstruction shouldbe compared to parametric maps obtained by gold standard parametric map-ping (Balsiger et al., 2018). However, while this is possible for FF (e.g., using3-point Dixon (Glover and Schneider, 1991)), there exists no MR sequence forT1 H2O mapping in fatty infiltrated tissue. Second, the optimal MRF data han-dling is still subject to further research. We have investigated several variants(see Section 1.3 of the supplementary material), but there are certainly openquestions such as the complex-valued nature of the MRF data, e.g., reconstruc-tion by complex-valued CNN (Virtue et al., 2017; Trabelsi et al., 2018). Also,the modeling of the temporal domain, e.g., by RNNs as presented by Oksuz et al.(2019) or by 3-D CNNs, needs further research. Finally, we can not make anystatements of the performance on other MRF sequences. Ideally, a completelyindependent dataset acquired with another MRF sequence and with other dis-eases should be available to demonstrate applicability among MRF sequences.In conclusion, we proposed an adaptable CNN for accurate and fast recon-struction of parametric maps from MRF. We demonstrated that incorporating aspatial neighborhood of fingerprints during the reconstruction is beneficial andthat we achieved excellent reconstruction accuracy and robustness to heteroge-neous patient data. The proposed method could enable MRF beyond clinicalresearch studies.
Acknowledgments
This research was supported by the Swiss National Science Foundation(SNSF) under grant number 184273. The authors thank the Nvidia Corpo-ration for their GPU donation and acknowledge the valuable discussions withPierre-Yves Baudin.
References
Assl¨ander, J., Cloos, M.A., Knoll, F., Sodickson, D.K., Hennig, J., Lattanzi, R.,2018. Low Rank Alternating Direction Method of Multipliers Reconstructionfor MR Fingerprinting. Magnetic Resonance in Medicine 79, 83–96. doi: .Balsiger, F., Scheidegger, O., Carlier, P.G., Marty, B., Reyes, M., 2019. On theSpatial and Temporal Influence for the Reconstruction of Magnetic ResonanceFingerprinting, in: Cardoso, M.J., Feragen, A., Glocker, B., Konukoglu, E.,Oguz, I., Unal, G., Vercauteren, T. (Eds.), Proceedings of The 2nd Interna-tional Conference on Medical Imaging with Deep Learning, PMLR, London.pp. 27–38. 20alsiger, F., Shridhar Konar, A., Chikop, S., Chandran, V., Scheidegger, O.,Geethanath, S., Reyes, M., 2018. Magnetic Resonance Fingerprinting Re-construction via Spatiotemporal Convolutional Neural Networks, in: Knoll,F., Maier, A., Rueckert, D. (Eds.), Machine Learning for Medical Image Re-construction. Springer, Cham. volume 11074 of
Lecture Notes in ComputerScience , pp. 39–46. doi: .Breiman, L., 2001a. Random Forests. Machine Learning 45, 5–32. doi: .Breiman, L., 2001b. Statistical Modeling: The Two Cultures. Statistical Science16, 199–231. doi: .Carlier, P.G., Marty, B., Scheidegger, O., Loureiro de Sousa, P., Baudin, P.Y.,Snezhko, E., Vlodavets, D., 2016. Skeletal Muscle Quantitative NuclearMagnetic Resonance Imaging and Spectroscopy as an Outcome Measure forClinical Trials. Journal of Neuromuscular Diseases 3, 1–28. doi: .Cauley, S.F., Setsompop, K., Ma, D., Jiang, Y., Ye, H., Adalsteinsson, E.,Griswold, M.A., Wald, L.L., 2015. Fast Group Matching for MR Fin-gerprinting Reconstruction. Magnetic Resonance in Medicine 74, 523–528.doi: .Cohen, O., Rosen, M.S., 2017. Algorithm comparison for schedule optimizationin MR fingerprinting. Magnetic Resonance Imaging 41, 15–21. doi: .Cohen, O., Zhu, B., Rosen, M.S., 2018. MR fingerprinting Deep RecOnstructionNEtwork (DRONE). Magnetic Resonance in Medicine 80, 885–894. doi: .Davies, M., Puy, G., Vandergheynst, P., Wiaux, Y., 2014. A Compressed Sens-ing Framework for Magnetic Resonance Fingerprinting. SIAM Journal onImaging Sciences 7, 2623–2656. doi: .Fang, Z., Chen, Y., Liu, M., Xiang, L., Zhang, Q., Wang, Q., Lin, W., Shen, D.,2019. Deep Learning for Fast and Spatially-Constrained Tissue Quantificationfrom Highly-Accelerated Data in Magnetic Resonance Fingerprinting. IEEETransactions on Medical Imaging 38, 2364–2374. doi: .Fang, Z., Chen, Y., Liu, M., Zhan, Y., Lin, W., Shen, D., 2018. Deep Learn-ing for Fast and Spatially-Constrained Tissue Quantification from Highly-Undersampled Data in Magnetic Resonance Fingerprinting (MRF), in: Shi,Y., Suk, H.I., Liu, M. (Eds.), Machine Learning in Medical Imaging. Springer,Cham. volume 11046 of
Lecture Notes in Computer Science , pp. 398–405.doi: .21essler, J., Sutton, B., 2003. Nonuniform Fast Fourier Transforms Using Min-Max Interpolation. IEEE Transactions on Signal Processing 51, 560–574.doi: .Fisher, A., Rudin, C., Dominici, F., 2019. All Models are Wrong, but Manyare Useful: Learning a Variable’s Importance by Studying an Entire Classof Prediction Models Simultaneously. Journal of Machine Learning Research20, 1–81.Glorot, X., Bordes, A., Bengio, Y., 2011. Deep Sparse Rectifier Neural Networks,in: Gordon, G., Dunson, D., Dud´ık, M. (Eds.), International Conference onArtificial Intelligence and Statistics, PMLR, Fort Lauderdale. pp. 315–323.Glover, G.H., Schneider, E., 1991. Three-point Dixon technique for true wa-ter/fat decomposition with B0 inhomogeneity correction. Magnetic Resonancein Medicine 18, 371–383. doi: .Golbabaee, M., Chen, D., G´omez, P.A., Menzel, M.I., Davies, M.E., 2019. Ge-ometry of Deep Learning for Magnetic Resonance Fingerprinting, in: IEEE In-ternational Conference on Acoustics, Speech and Signal Processing (ICASSP),IEEE. pp. 7825–7829. doi: .G´omez, P.A., Molina-Romero, M., Ulas, C., Bounincontri, G., Sperl, J.I., Jones,D.K., Menzel, M.I., Menze, B.H., 2016. Simultaneous Parameter Mapping,Modality Synthesis, and Anatomical Labeling of the Brain with MR Finger-printing, in: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells,W. (Eds.), Medical Image Computing and Computer-Assisted Intervention– MICCAI 2016, Springer International Publishing, Cham. pp. 579–586.doi: .Hoppe, E., K¨orzd¨orfer, G., W¨urfl, T., Wetzl, J., Lugauer, F., Pfeuffer, J., Maier,A., 2017. Deep Learning for Magnetic Resonance Fingerprinting: A NewApproach for Predicting Quantitative Parameter Values from Time Series.,in: R¨ohrig, R., Timmer, A., Binder, H., Sax, U. (Eds.), German MedicalData Sciences: Visions and Bridges, Oldenburg, Oldenburg. pp. 202–206.doi: .Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q., 2017. Densely Con-nected Convolutional Networks, in: IEEE Conference on Computer Visionand Pattern Recognition (CVPR), IEEE. pp. 2261–2269. doi: .Ioffe, S., Szegedy, C., 2015. Batch Normalization: Accelerating Deep NetworkTraining by Reducing Internal Covariate Shift, in: Bach, F., Blei, D. (Eds.),International Conference on Machine Learning, PMLR, Lille. pp. 448–456.Kingma, D.P., Ba, J.L., 2015. Adam: A Method for Stochastic Optimization, in:International Conference on Learning Representations. arXiv:1412.6980 .22noll, F., Hammernik, K., Kobler, E., Pock, T., Recht, M.P., Sodickson, D.K.,2019. Assessment of the generalization of learned image reconstruction andthe potential for transfer learning. Magnetic Resonance in Medicine 81, 116–128. doi: .Lee, P.K., Watkins, L.E., Anderson, T.I., Buonincontri, G., Hargreaves, B.A.,2019. Flexible and efficient optimization of quantitative sequences using au-tomatic differentiation of Bloch simulations. Magnetic Resonance in Medicine82, 1438–1451. doi: .Lima da Cruz, G., Bustin, A., Jaubert, O., Schneider, T., Botnar, R.M., Prieto,C., 2019. Sparsity and locally low rank regularization for MR fingerprinting.Magnetic Resonance in Medicine 81, 3530–3543. doi: .Ma, D., Gulani, V., Seiberlich, N., Liu, K., Sunshine, J.L., Duerk, J.L., Gris-wold, M.A., 2013. Magnetic resonance fingerprinting. Nature 495, 187–192.doi: .Marty, B., Carlier, P.G., 2019a. MR fingerprinting for water T1 and fat frac-tion quantification in fat infiltrated skeletal muscles. Magnetic Resonance inMedicine 83, 621–634. doi: .Marty, B., Carlier, P.G., 2019b. Physiological and pathological skeletal muscleT1 changes quantified using a fast inversion-recovery radial NMR imagingsequence. Scientific Reports 9, 6852. doi: .Mazor, G., Weizman, L., Tal, A., Eldar, Y.C., 2018. Low-rank magnetic reso-nance fingerprinting. Medical Physics 45, 4066–4084. doi: .McGivney, D.F., Boyacolu, R., Jiang, Y., Poorman, M.E., Seiberlich, N., Gu-lani, V., Keenan, K.E., Griswold, M.A., Ma, D., 2019. Magnetic resonancefingerprinting review part 2: Technique and directions. Journal of MagneticResonance Imaging 51, 993–1007. doi: .McGivney, D.F., Pierre, E., Ma, D., Jiang, Y., Saybasili, H., Gulani, V., Gris-wold, M.A., 2014. SVD Compression for Magnetic Resonance Fingerprintingin the Time Domain. IEEE Transactions on Medical Imaging 33, 2311–2322.doi: .Oksuz, I., Cruz, G., Clough, J., Bustin, A., Fuin, N., Botnar, R.M., Prieto,C., King, A.P., Schnabel, J.A., 2019. Magnetic Resonance Fingerprinting us-ing Recurrent Neural Networks, in: International Symposium on BiomedicalImaging, IEEE. pp. 1537–1540. doi: .Paoletti, M., Pichiecchio, A., Cotti Piccinelli, S., Tasca, G., Berardinelli, A.L.,Padovani, A., Filosto, M., 2019. Advances in Quantitative Imaging of Geneticand Acquired Myopathies: Clinical Applications and Perspectives. Frontiersin Neurology 10, 78. doi: .23ierre, E.Y., Ma, D., Chen, Y., Badve, C., Griswold, M.A., 2016. MultiscaleReconstruction for MR Fingerprinting. Magnetic Resonance in Medicine 75,2481–2492. doi: .Poorman, M.E., Martin, M.N., Ma, D., McGivney, D.F., Gulani, V., Griswold,M.A., Keenan, K.E., 2019. Magnetic resonance fingerprinting Part 1: Po-tential uses, current challenges, and recommendations. Journal of MagneticResonance Imaging 51, 675–692. doi: .Song, P., Eldar, Y.C., Mazor, G., Rodrigues, M.R., 2019. HYDRA: HybridDeep Magnetic Resonance Fingerprinting. Medical Physics 46, 4951–4969.doi: , arXiv:1902.02882 .Trabelsi, C., Bilaniuk, O., Zhang, Y., Serdyuk, D., Subramanian, S., Santos,J.F., Mehri, S., Rostamzadeh, N., Bengio, Y., Pal, C.J., 2018. Deep Com-plex Networks, in: International Conference on Learning Representations. arXiv:1705.09792 .Virtue, P., Yu, S.X., Lustig, M., 2017. Better than Real: Complex-valued NeuralNets for MRI Fingerprinting, in: IEEE International Conference on ImageProcessing (ICIP), IEEE. pp. 3953–3957. doi: .Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E., 2004. Image Quality Assess-ment: From Error Visibility to Structural Similarity. IEEE Transactions onImage Processing 13, 600–612. doi: .Zbontar, J., Knoll, F., Sriram, A., Muckley, M.J., Bruno, M., Defazio, A., Par-ente, M., Geras, K.J., Katsnelson, J., Chandarana, H., Zhang, Z., Drozdzal,M., Romero, A., Rabbat, M., Vincent, P., Pinkerton, J., Wang, D., Yakubova,N., Owens, E., Zitnick, C.L., Recht, M.P., Sodickson, D.K., Lui, Y.W., 2018.fastMRI: An Open Dataset and Benchmarks for Accelerated MRI. arXivpreprint arXiv:1811.08839 arXiv:1811.08839 .Zhao, B., Haldar, J.P., Liao, C., Ma, D., Jiang, Y., Griswold, M.A., Setsompop,K., Wald, L.L., 2019. Optimal Experiment Design for Magnetic ResonanceFingerprinting: Cram´er-Rao Bound Meets Spin Dynamics. IEEE Transac-tions on Medical Imaging 38, 844–861. doi: .Zhao, B., Setsompop, K., Adalsteinsson, E., Gagoski, B., Ye, H., Ma, D.,Jiang, Y., Ellen Grant, P., Griswold, M.A., Wald, L.L., 2018. Improved Mag-netic Resonance Fingerprinting Reconstruction With Low-Rank and SubspaceModeling. Magnetic Resonance in Medicine 79, 933–942. doi: . 24 igure 7: Influence of the temporal frames on the parametric map reconstruction. (firstrow) The MRF T1-FF sequence parameters and the importance of each temporal frame for(second row) FF, (third row) T1 H2O , (fourth row) T1 fat , (fifth row) ∆f, and (sixth row) B1reconstruction. igure 8: Robustness of the proposed method to heterogeneous morphometric variationsdepending on the number of training subjects. The CNN was trained with 5, 15, 25, 40, 55,70, and 85 subjects randomly taken from the training split of 94 subjects. The error barsindicate the standard deviation of five random runs. Dictionary matching ProposedFF T1
H2O (ms)Error Sh o u l d er A bd o m i n a l P e l v i s Dictionary matching Proposed Error