Pradeep Reddy Raamana

Assessing and tuning brain decoders: cross-validation, caveats, and guidelines

ABSTRACT Decoding, i.e. prediction from brain images or signals, calls for empirical evaluation of its predictive power. Such evaluation is achieved via cross‐validation, a method also used to tune decoders hyper‐parameters. This paper is a review on cross‐validation procedures for decoding in neuroimaging. It includes a didactic overview of the relevant theoretical considerations. Practical aspects are highlighted with an extensive empirical study of the common decoders in within‐ and across‐subject predictions, on multiple datasets –anatomical and functional MRI and MEG– and simulations. Theory and experiments outline that the popular “leave‐one‐out” strategy leads to unstable and biased estimates, and a repeated random splits method should be preferred. Experiments outline the large error bars of cross‐validation in neuroimaging settings: typical confidence intervals of 10%. Nested cross‐validation can tune decoders parameters while avoiding circularity bias. However we find that it can be favorable to use sane defaults, in particular for non‐sparse decoders. Graphical abstract Figure. No Caption available. HighlightsWe give a primer on cross‐validation to measure decoders predictive power.We assess on many datasets its practical use for decoding selection and tuning.Cross‐validation displays large confidence intervals, in particular leave one out.Default parameters on standard decoders can outperform parameter tuning.

Automated detection of amnestic mild cognitive impairment in community-dwelling elderly adults: a combined spatial atrophy and white matter alteration approach.

Amnestic mild cognitive impairment (aMCI) is a syndrome widely considered to be prodromal Alzheimers disease. Accurate diagnosis of aMCI would enable earlier treatment, and could thus help minimize the prevalence of Alzheimers disease. The aim of the present study was to evaluate a magnetic resonance imaging-based automated classification schema for identifying aMCI. This was carried out in a sample of community-dwelling adults aged 70-90 years old: 79 with a clinical diagnosis of aMCI and 204 who were cognitively normal. Our schema was novel in using measures of both spatial atrophy, derived from T1-weighted images, and white matter alterations, assessed with diffusion tensor imaging (DTI) tract-based spatial statistics (TBSS). Subcortical volumetric features were extracted using a FreeSurfer-initialized Large Deformation Diffeomorphic Metric Mapping (FS+LDDMM) segmentation approach, and fractional anisotropy (FA) values obtained for white matter regions of interest. Features were ranked by their ability to discriminate between aMCI and normal cognition, and a support vector machine (SVM) selected an optimal feature subset that was used to train SVM classifiers. As evaluated via 10-fold cross-validation, the classification performance characteristics achieved by our schema were: accuracy, 71.09%; sensitivity, 51.96%; specificity, 78.40%; and area under the curve, 0.7003. Additionally, we identified numerous socio-demographic, lifestyle, health and other factors potentially implicated in the misclassification of individuals by our schema and those previously used by others. Given its high level of performance, our classification schema could facilitate the early detection of aMCI in community-dwelling elderly adults.

BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods

The rate of progress in human neurosciences is limited by the inability to easily apply a wide range of analysis methods to the plethora of different datasets acquired in labs around the world. In this work, we introduce a framework for creating, testing, versioning and archiving portable applications for analyzing neuroimaging data organized and described in compliance with the Brain Imaging Data Structure (BIDS). The portability of these applications (BIDS Apps) is achieved by using container technologies that encapsulate all binary and other dependencies in one convenient package. BIDS Apps run on all three major operating systems with no need for complex setup and configuration and thanks to the comprehensiveness of the BIDS standard they require little manual user input. Previous containerized data processing solutions were limited to single user environments and not compatible with most multi-tenant High Performance Computing systems. BIDS Apps overcome this limitation by taking advantage of the Singularity container technology. As a proof of concept, this work is accompanied by 22 ready to use BIDS Apps, packaging a diverse set of commonly used neuroimaging algorithms.

Three-Class Differential Diagnosis among Alzheimer Disease, Frontotemporal Dementia, and Controls

Biomarkers derived from brain magnetic resonance (MR) imaging have promise in being able to assist in the clinical diagnosis of brain pathologies. These have been used in many studies in which the goal has been to distinguish between pathologies such as Alzheimer’s disease and healthy aging. However, other dementias, in particular, frontotemporal dementia, also present overlapping pathological brain morphometry patterns. Hence, a classifier that can discriminate morphometric features from a brain MRI from the three classes of normal aging, Alzheimer’s disease (AD), and frontotemporal dementia (FTD) would offer considerable utility in aiding in correct group identification. Compared to the conventional use of multiple pair-wise binary classifiers that learn to discriminate between two classes at each stage, we propose a single three-way classification system that can discriminate between three classes at the same time. We present a novel classifier that is able to perform a three-class discrimination test for discriminating among AD, FTD, and normal controls (NC) using volumes, shape invariants, and local displacements (three features) of hippocampi and lateral ventricles (two structures times two hemispheres individually) obtained from brain MR images. In order to quantify its utility in correct discrimination, we optimize the three-class classifier on a training set and evaluate its performance using a separate test set. This is a novel, first-of-its-kind comparative study of multiple individual biomarkers in a three-class setting. Our results demonstrate that local atrophy features in lateral ventricles offer the potential to be a biomarker in discriminating among AD, FTD, and NC in a three-class setting for individual patient classification.

Thickness network features for prognostic applications in dementia

Regional analysis of cortical thickness has been studied extensively in building imaging biomarkers for early detection of Alzheimers disease but not its interregional covariation of thickness. We present novel features based on the inter-regional covariation of cortical thickness. Initially, the cortical labels of each subject are partitioned into small patches (graph nodes) by spatial k-means clustering. A graph is then constructed by establishing a link between 2 nodes if the difference in thickness between the nodes is below a certain threshold. From this binary graph, a thickness network is computed using nodal degree, betweenness, and clustering coefficient measures. Fusing them with multiple kernel learning, it is observed that thickness network features discriminate mild cognitive impairment (MCI) converters from controls (CN) with an area under curve (AUC) of 0.83, 74% sensitivity and 76% specificity on a large subset obtained from the Alzheimers Disease Neuroimaging Initiative data set. A comparison of predictive utility in Alzheimers disease and/or CN classification (AUC of 0.92, 80% sensitivity [SENS] and 90% specificity [SPEC]), in discriminating CN from MCI (converters and nonconverters combined; AUC of 0.75, SENS and SPEC of 64% and 73%, respectively) and in discriminating between MCI nonconverters and MCI converters (AUC of 0.68, SENS and SPEC of 65% and 64%) is also presented. ThickNet features as defined here are novel, can be derived from a single magnetic resonance imaging scan, and demonstrate the potential for the computer-aided prognostic applications.

Novel ThickNet features for the discrimination of amnestic MCI subtypes

Background Amnestic mild cognitive impairment (aMCI) is considered to be a transitional stage between healthy aging and Alzheimers disease (AD), and consists of two subtypes: single-domain aMCI (sd-aMCI) and multi-domain aMCI (md-aMCI). Individuals with md-aMCI are found to exhibit higher risk of conversion to AD. Accurate discrimination among aMCI subtypes (sd- or md-aMCI) and controls could assist in predicting future decline. Methods We apply our novel thickness network (ThickNet) features to discriminate md-aMCI from healthy controls (NC). ThickNet features are extracted from the properties of a graph constructed from inter-regional co-variation of cortical thickness. We fuse these ThickNet features using multiple kernel learning to form a composite classifier. We apply the proposed ThickNet classifier to discriminate between md-aMCI and NC, sd-aMCI and NC and; and also between sd-aMCI and md-aMCI, using baseline T1 MR scans from the Sydney Memory and Ageing Study. Results ThickNet classifier achieved an area under curve (AUC) of 0.74, with 70% sensitivity and 69% specificity in discriminating md-aMCI from healthy controls. The same classifier resulted in AUC = 0.67 and 0.67 for sd-aMCI/NC and sd-aMCI/md-aMCI classification experiments respectively. Conclusions The proposed ThickNet classifier demonstrated potential for discriminating md-aMCI from controls, and in discriminating sd-aMCI from md-aMCI, using cortical features from baseline MRI scan alone. Use of the proposed novel ThickNet features demonstrates significant improvements over previous experiments using cortical thickness alone. This result may offer the possibility of early detection of Alzheimers disease via improved discrimination of aMCI subtypes.

Comparison of four shape features for detecting hippocampal shape changes in early Alzheimer's

We compare four methods for generating shape-based features from 3D binary images of the hippocampus for use in group discrimination and classification. The first method we investigate is based on decomposing the hippocampal binary segmentation onto an orthonormal basis of spherical harmonics, followed by computation of shape invariants by tensor contraction using the Clebsch–Gordan coefficients. The second method we investigate is based on the classical 3D moment invariants; these are a special case of the spherical harmonics-based tensor invariants. The third method is based on solving the Helmholtz equation on the geometry of the binary hippocampal segmentation, and construction of shape-descriptive features from the eigenvalues of the Fourier-like modes of the geometry represented by the Laplacian eigenfunctions. The fourth method investigates the use of initial momentum obtained from the large-deformation diffeomorphic metric mapping method as a shape feature. Each of these shape features is tested for group differences in the control (Clinical Dementia Rating Scale CDR 0) and the early (very mild) Alzheimers (CDR 0.5) population. Classification of individual shapes is performed via a linear support vector machine based classifer with leave-one-out cross validation to test for overall performance. These experiments show that all of these feature computation approaches gave stable and reasonable classification results on the same database, and with the same classifier. The best performance was achieved with the shape-features constructed from large-deformation diffeomorphic metric mapping-based initial momentum.

Impact of spatial scale and edge weight on predictive power of cortical thickness networks

Network-level analysis based on anatomical covariance (cortical thickness) has been gaining increasing popularity recently. However, there has not been a systematic study of the impact of spatial scale and edge definitions on predictive performance. In order to obtain a clear understanding of relative performance, there is a need for systematic comparison. In this study, we present a histogram-based approach to construct subject-wise weighted networks that enable a principled comparison across different methods of network analysis. We design several weighted networks based on two large publicly available datasets and perform a robust evaluation of their predictive power under three levels of separability. One of the interesting insights include the robust predictive power resulting from lack of significant impact of changes in nodal size (spatial scale) among the three classification experiments. We also release an open source python package to enable others to implement presented network feature extraction algorithm in their research.Network-level analysis based on anatomical, pairwise similarities (e.g., cortical thickness) has been gaining increasing attention recently. However, there has not been a systematic study of the impact of spatial scale and edge definitions on predictive performance. In order to obtain a clear understanding of relative performance, there is a need for systematic comparison. In this study, we present a histogram-based approach to construct subject-wise weighted networks that enable a principled comparison across different methods of network analysis. We design several weighted networks based on three large publicly available datasets and perform a robust evaluation of their predictive power under four levels of separability. An interesting insight generated is that changes in nodal size (spatial scale) have no significant impact on predictive power among the three classification experiments and two disease cohorts studied, i.e., mild cognitive impairment and Alzheimer’s disease from ADNI, and Autism from the ABIDE dataset. We also release an open source python package called graynet to enable others to implement the novel network feature extraction algorithm, which is applicable to other modalities as well (due to its domain- and feature-agnostic nature) in diverse applications of connectivity research. In addition, the findings from the ADNI dataset are replicated in the AIBL dataset using an open source machine learning tool called neuropredict.Network-level analysis based on anatomical, pairwise similarities (e.g., cortical thickness) has been gaining increasing attention recently. However, there has not been a systematic study of the impact of spatial scale and edge definitions on predictive performance. In order to obtain a clear understanding of relative performance, there is a need for systematic comparison. In this study, we present a histogram-based approach to construct subject-wise weighted networks that enable a principled comparison across different methods of network analysis. We design several weighted networks based on three large publicly available datasets and perform a robust evaluation of their predictive power under four levels of separability. An interesting insight generated is that changes in nodal size (spatial scale) have no significant impact on predictive power among the three classification experiments and two disease cohorts studied, i.e., mild cognitive impairment and Alzheimer9s disease from ADNI, and Autism from the ABIDE dataset. We also release an open source python package called graynet to enable others to implement the novel network feature extraction algorithm, which is applicable to other modalities as well (due to its domain- and feature-agnostic nature) in diverse applications of connectivity research. In addition, the findings from the ADNI dataset are replicated in the AIBL dataset using an open source machine learning tool called neuropredict.

The Sub-Classification of Amnestic Mild Cognitive Impairment Using MRI-Based Cortical Thickness Measures.

Background: Amnestic mild cognitive impairment (aMCI) is considered to be the transitional stage between healthy aging and Alzheimer’s disease (AD). Moreover, aMCI individuals with additional impairment in one or more non-memory cognitive domains are at higher risk of conversion to AD. Hence accurate identification of the sub-types of aMCI would enable earlier detection of individuals progressing to AD. Methods: We examine the group differences in cortical thickness between single-domain and multiple-domain sub-types of aMCI, and as well as with respect to age-matched controls in a well-balanced cohort from the Sydney Memory and Aging Study. In addition, the diagnostic value of cortical thickness in the sub-classification of aMCI as well as from normal controls using support vector machine (SVM) classifier is evaluated, using a novel cross-validation technique that can handle class-imbalance. Results: This study revealed an increased, as well as a wider spread, of cortical thinning in multiple-domain aMCI compared to single-domain aMCI. The best performances of the classifier for the pairs (1) single-domain aMCI and normal controls, (2) multiple-domain aMCI and normal controls, and (3) single and multiple-domain aMCI were AUCu2009=u20090.52, 0.66, and 0.54, respectively. The accuracy of the classifier for the three pairs was just over 50% exhibiting low specificity (44–60%) and similar sensitivity (53–68%). Conclusion: Analysis of group differences added evidence to the hypothesis that multiple-domain aMCI is a later stage of AD compared to single-domain aMCI. The classification results show that discrimination among single, multiple-domain sub-types of aMCI and normal controls is limited using baseline cortical thickness measures.

Thickness NETwork (ThickNet) Features for the Detection of Prodromal AD

Regional analysis of cortical thickness has been studied extensively in building imaging biomarkers for early detection of Alzheimers disease (AD), but not its inter-regional covariation. We present novel features based on the inter-regional co-variation of cortical thickness. Initially the cortical labels of each patient is partitioned into small patches (graph nodes) by spatial k-means clustering. A graph is then constructed by establishing a link between two nodes if the difference in thickness between the nodes is below a certain threshold. From this binary graph, thickness network (ThickNet) features are computed using nodal degree, betweenness and clustering coefficient measures. Fusing them with multiple kernel learning, we demonstrate their potential for the detection of prodromal AD.