Stephen C. Strother

The Quantitative Evaluation of Functional Neuroimaging Experiments: The NPAIRS Data Analysis Framework

We introduce a data-analysis framework and performance metrics for evaluating and optimizing the interaction between activation tasks, experimental designs, and the methodological choices and tools for data acquisition, preprocessing, data analysis, and extraction of statistical parametric maps (SPMs). Our NPAIRS (nonparametric prediction, activation, influence, and reproducibility resampling) framework provides an alternative to simulations and ROC curves by using real PET and fMRI data sets to examine the relationship between prediction accuracy and the signal-to-noise ratios (SNRs) associated with reproducible SPMs. Using cross-validation resampling we plot training-test set predictions of the experimental design variables (e.g., brain-state labels) versus reproducibility SNR metrics for the associated SPMs. We demonstrate the utility of this framework across the wide range of performance metrics obtained from [(15)O]water PET studies of 12 age- and sex-matched data sets performing different motor tasks (8 subjects/set). For the 12 data sets we apply NPAIRS with both univariate and multivariate data-analysis approaches to: (1) demonstrate that this framework may be used to obtain reproducible SPMs from any data-analysis approach on a common Z-score scale (rSPM[Z]); (2) demonstrate that the histogram of a rSPM[Z] image may be modeled as the sum of a data-analysis-dependent noise distribution and a task-dependent, Gaussian signal distribution that scales monotonically with our reproducibility performance metric; (3) explore the relation between prediction and reproducibility performance metrics with an emphasis on bias-variance tradeoffs for flexible, multivariate models; and (4) measure the broad range of reproducibility SNRs and the significant influence of individual subjects. A companion paper describes learning curves for four of these 12 data sets, which describe an alternative mutual-information prediction metric and NPAIRS reproducibility as a function of training-set sizes from 2 to 18 subjects. We propose the NPAIRS framework as a validation tool for testing and optimizing methodological choices and tools in functional neuroimaging.

A Multivariate Analysis of Age-Related Differences in Default Mode and Task-Positive Networks across Multiple Cognitive Domains

We explored the effects of aging on 2 large-scale brain networks, the default mode network (DMN) and the task-positive network (TPN). During functional magnetic resonance imaging scanning, young and older participants carried out 4 visual tasks: detection, perceptual matching, attentional cueing, and working memory. Accuracy of performance was roughly matched at 80% across tasks and groups. Modulations of activity across conditions were assessed, as well as functional connectivity of both networks. Younger adults showed a broader engagement of the DMN and older adults a more extensive engagement of the TPN. Functional connectivity in the DMN was reduced in older adults, whereas the main pattern of TPN connectivity was equivalent in the 2 groups. Age-specific connectivity also was seen in TPN regions. Increased activity in TPN areas predicted worse accuracy on the tasks, but greater expression of a connectivity pattern associated with a right dorsolateral prefrontal TPN region, seen only in older adults, predicted better performance. These results provide further evidence for age-related differences in the DMN and new evidence of age differences in the TPN. Increased use of the TPN may reflect greater demand on cognitive control processes in older individuals that may be partially offset by alterations in prefrontal functional connectivity.

Generalizable patterns in neuroimaging: how many principal components?

Generalization can be defined quantitatively and can be used to assess the performance of principal component analysis (PCA). The generalizability of PCA depends on the number of principal components retained in the analysis. We provide analytic and test set estimates of generalization. We show how the generalization error can be used to select the number of principal components in two analyses of functional magnetic resonance imaging activation sets.

Machine Learning in Medical Imaging

This article will discuss very different ways of using machine learning that may be less familiar, and we will demonstrate through examples the role of these concepts in medical imaging. Although the term machine learning is relatively recent, the ideas of machine learning have been applied to medical imaging for decades, perhaps most notably in the areas of computer-aided diagnosis (CAD) and functional brain mapping. We will not attempt in this brief article to survey the rich literature of this field. Instead our goals will be 1) to acquaint the reader with some modern techniques that are now staples of the machine-learning field and 2) to illustrate how these techniques can be employed in various ways in medical imaging.

Reproducibility of BOLD-Based Functional MRI Obtained at 4 T

The reproducibility of activation patterns in the whole brain obtained by functional magnetic resonance imaging (fMRI) experiments at 4 Tesla was studied with a simple finger‐opposition task. Six subjects performed three runs in one session, and each run was analyzed separately with the t‐test as a univariate method and Fishers linear discriminant analysis as a multivariate method. Detrending with a first‐ and third‐order polynomial as well as logarithmic transformation as preprocessing steps for the t‐test were tested for their impact on reproducibility. Reproducibility across the whole brain was studied by using scatter plots of statistical values and calculating the correlation coefficient between pairs of activation maps. In order to compare reproducibility of “activated” voxels across runs, subjects and models, 2% of all voxels in the brain with the highest statistical values were classified as activated. The analysis of reproducible activated voxels was performed for the whole brain and within regions of interest. We found considerable variability in reproducibility across subjects, regions of interest, and analysis methods. The t‐test on the linear detrended data yielded better reproducibility than Fishers linear discriminant analysis, and therefore seems to be a robust although conservative method. Preliminary data indicate that these modeling results may be reversed by preprocessing to reduce respiratory and cardiac physiological noise effects. The reproducibility of both the position and number of activated voxels in the sensorimotor cortex was highest, while that of the supplementary motor area was much lower, with reproducibility of the cerebellum falling in between the other two areas. Hum. Brain Mapping 7:267–283, 1999.

Quantitative comparisons of image registration techniques based on high-resolution MRI of the brain.

Objective A variety of methods for matching intrasubject MRI-MRI, PET-PET, or MRI-PET image pairs have been proposed. Based on the rigid body transformations needed to align pairs of high-resolution MRI scans and/or simulated PET scans (derived from these MRI scans), we obtained general comparisons of four intrasubject image registration techniques: Talairach coordinates, head and hat, equivalent internal points, and ratio image uniformity. In addition, we obtained a comparison of stereotaxic Z frames with a customized head mold for MRI-MRI image pairs. Materials and Methods and Results Each technique was quantitatively evaluated using the mean and maximum voxel registration errors for matched voxel pairs within the brain volumes being registered. Conclusion We conclude that fiducial markers such as stereotaxic Z frames that are not rigidly fixed to a patients skull are inaccurate compared with other registration techniques, Talairach coordinate transformations provide surprisingly good registration, and minimizing the variance of MRI-MRI, PET-PET, or MRI-PET ratio images provides significantly better registration than all other techniques tested. Registration optimization based on measurement of the similarity of spatial distributions of voxel values is superior to techniques that do not use such information.

The Evaluation of Preprocessing Choices in Single-Subject BOLD fMRI Using NPAIRS Performance Metrics

This work proposes an alternative to simulation-based receiver operating characteristic (ROC) analysis for assessment of fMRI data analysis methodologies. Specifically, we apply the rapidly developing nonparametric prediction, activation, influence, and reproducibility resampling (NPAIRS) framework to obtain cross-validation-based model performance estimates of prediction accuracy and global reproducibility for various degrees of model complexity. We rely on the concept of an analysis chain meta-model in which all parameters of the preprocessing steps along with the final statistical model are treated as estimated model parameters. Our ROC analog, then, consists of plotting prediction vs. reproducibility results as curves of model complexity for competing meta-models. Two theoretical underpinnings are crucial to utilizing this new validation technique. First, we explore the relationship between global signal-to-noise and our reproducibility estimates as derived previously. Second, we submit our model complexity curves in the prediction versus reproducibility space as reflecting classic bias-variance tradeoffs. Among the particular analysis chains considered, we found little impact in performance metrics with alignment, some benefit with temporal detrending, and greatest improvement with spatial smoothing.

Abnormal network connectivity in frontotemporal dementia: Evidence for prefrontal isolation

INTRODUCTION Degraded social function, disinhibition, and stereotypy are defining characteristics of frontotemporal dementia (FTD), manifesting in both the behavioral variant of frontotemporal dementia (bvFTD) and semantic dementia (SD) subtypes. Recent neuroimaging research also associates FTD with alterations in the brains intrinsic connectivity networks. The present study explored the relationship between neural network connectivity and specific behavioral symptoms in FTD. METHODS Resting-state functional magnetic resonance imaging was employed to investigate neural network changes in bvFTD and SD. We used independent components analysis (ICA) to examine changes in frontolimbic network connectivity, as well as several metrics of local network strength, such as the fractional amplitude of low-frequency fluctuations, regional homogeneity, and seed-based functional connectivity. For each analysis, we compared each FTD subgroup to healthy controls, characterizing general and subtype-unique network changes. The relationship between abnormal connectivity in FTD and behavior disturbances was explored. RESULTS Across multiple analytic approaches, both bvFTD and SD were associated with disrupted frontolimbic connectivity and elevated local connectivity within the prefrontal cortex. Even after controlling for structural atrophy, prefrontal hyperconnectivity was robustly associated with apathy scores. Frontolimbic disconnection was associated with lower disinhibition scores, suggesting that abnormal frontolimbic connectivity contributes to positive symptoms in dementia. Unique to bvFTD, stereotypy was associated with elevated default network connectivity in the right angular gyrus. The behavioral variant was also associated with marginally higher apathy scores and a more diffuse pattern of prefrontal hyperconnectivity than SD. CONCLUSIONS The present findings support a theory of FTD as a disorder of frontolimbic disconnection leading to unconstrained prefrontal connectivity. Prefrontal hyperconnectivity may represent a compensatory response to the absence of affective feedback during the planning and execution of behavior. Increased reliance upon prefrontal processes in isolation from subcortical structures appears to be maladaptive and may drive behavioral withdrawal that is commonly observed in later phases of neurodegeneration.

Practical tradeoffs between noise, quantitation, and number of iterations for maximum likelihood-based reconstructions

Emission computerised tomography images reconstructed using a maximum likelihood-expectation maximization (ML)-based method with different reconstruction kernels and 1-200 iterations are compared to images reconstructed using filtered backprojection (FBP). ML-based reconstructions using a single pixel (SP) kernel with or without a sieve filter show no quantitative advantage over FBP except in the background where a reduction of noise is possible if the number of iterations is kept small (<50). ML-based reconstructions using a Gaussian kernel with a multipixel full-width-at-half-maximum (FWHM) and a large number of iterations (200) require a sieve filtering step to reduce the noise and contrast overshoot in the final images. These images have some small quantitative advantages over FBP depending on the structures being imaged. It is demonstrated that a feasibility stopping criterion controls the noise in a reconstructed image, but is insensitive to quantitation errors, and that the use of an appropriate overrelaxation parameter can accelerate the convergence of the ML-based method during the iterative process without quantitative instabilities.

Enhancing the multivariate signal of [/sup 15/O] water PET studies with a new nonlinear neuroanatomical registration algorithm [MRI application]

This paper addresses the problem of neuro-anatomical registration across individuals for functional [15O] water PET activation studies. A new algorithm for three-dimensional (3-D) nonlinear structural registration (warping) of MR scans is presented. The method performs a hierarchically scaled search for a displacement field, maximizing one of several voxel similarity measures derived from the two-dimensional (2-D) histogram of matched image intensities, subject to a regularizer that ensures smoothness of the displacement field. The effect of the nonlinear structural registration is studied when it is computed on anatomical MR scans and applied to coregistered [15O] water PET scans from the same subjects: in this experiment, a study of visually guided saccadic eye movements. The performance of the nonlinear warp is evaluated using multivariate functional signal and noise measures. These measures prove to be useful for comparing different intersubject registration approaches, e.g., affine versus nonlinear. A comparison of 12-parameter affine registration versus non-linear registration demonstrates that the proposed nonlinear method increases the number of voxels retained in the cross-subject mask. We demonstrate that improved structural registration may result in an improved multivariate functional signal-to-noise ratio (SNR). Furthermore, registration of PET scans using the 12-parameter affine transformations that align the coregistered MR images does not improve registration, compared to 12-parameter affine alignment of the PET images directly.