Maciej A. Mazurowski

Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance.

This study investigates the effect of class imbalance in training data when developing neural network classifiers for computer-aided medical diagnosis. The investigation is performed in the presence of other characteristics that are typical among medical data, namely small training sample size, large number of features, and correlations between features. Two methods of neural network training are explored: classical backpropagation (BP) and particle swarm optimization (PSO) with clinically relevant training criteria. An experimental study is performed using simulated data and the conclusions are further validated on real clinical data for breast cancer diagnosis. The results show that classifier performance deteriorates with even modest class imbalance in the training data. Further, it is shown that BP is generally preferable over PSO for imbalanced training data especially with small data sample and large number of features. Finally, it is shown that there is no clear preference between oversampling and no compensation approach and some guidance is provided regarding a proper selection.

Radiogenomic Analysis of Breast Cancer: Luminal B Molecular Subtype Is Associated with Enhancement Dynamics at MR Imaging

PURPOSE To investigate associations between breast cancer molecular subtype and semiautomatically extracted magnetic resonance (MR) imaging features. MATERIALS AND METHODS Imaging and genomic data from the Cancer Genome Atlas and the Cancer Imaging Archive for 48 patients with breast cancer from four institutions in the United States were used in this institutional review board approval-exempt study. Computer vision algorithms were applied to extract 23 imaging features from lesions indicated by a breast radiologist on MR images. Morphologic, textural, and dynamic features were extracted. Molecular subtype was determined on the basis of genomic analysis. Associations between the imaging features and molecular subtype were evaluated by using logistic regression and likelihood ratio tests. The analysis controlled for the age of the patients, their menopausal status, and the orientation of the MR images (sagittal vs axial). RESULTS There is an association (P = .0015) between the luminal B subtype and a dynamic contrast material-enhancement feature that quantifies the relationship between lesion enhancement and background parenchymal enhancement. Cancers with a higher ratio of lesion enhancement rate to background parenchymal enhancement rate are more likely to be luminal B subtype. CONCLUSION The luminal B subtype of breast cancer is associated with MR imaging features that relate the enhancement dynamics of the tumor and the background parenchyma.

Imaging descriptors improve the predictive power of survival models for glioblastoma patients

BACKGROUND Because effective prediction of survival time can be highly beneficial for the treatment of glioblastoma patients, the relationship between survival time and multiple patient characteristics has been investigated. In this paper, we investigate whether the predictive power of a survival model based on clinical patient features improves when MRI features are also included in the model. METHODS The subjects in this study were 82 glioblastoma patients for whom clinical features as well as MR imaging exams were made available by The Cancer Genome Atlas (TCGA) and The Cancer Imaging Archive (TCIA). Twenty-six imaging features in the available MR scans were assessed by radiologists from the TCGA Glioma Phenotype Research Group. We used multivariate Cox proportional hazards regression to construct 2 survival models: one that used 3 clinical features (age, gender, and KPS) as the covariates and 1 that used both the imaging features and the clinical features as the covariates. Then, we used 2 measures to compare the predictive performance of these 2 models: area under the receiver operating characteristic curve for the 1-year survival threshold and overall concordance index. To eliminate any positive performance estimation bias, we used leave-one-out cross-validation. RESULTS The performance of the model based on both clinical and imaging features was higher than the performance of the model based on only the clinical features, in terms of both area under the receiver operating characteristic curve (P < .01) and the overall concordance index (P < .01). CONCLUSIONS Imaging features assessed using a controlled lexicon have additional predictive value compared with clinical features when predicting survival time in glioblastoma patients.

Radiogenomics: What It Is and Why It Is Important

In recent years, a new direction in cancer research has emerged that focuses on the relationship between imaging phenotypes and genomics. This direction is referred to as radiogenomics or imaging genomics. The question that subsequently arises is: What is the practical significance of elucidating this relationship in improving cancer patient outcomes. In this article, I address this question. Although I discuss some limitations of the radiogenomic approach, and describe scenarios in which radiogenomic analysis might not be the best choice, I also argue that radiogenomics will play a significant practical role in cancer research. Specifically, I argue that the significance of radiogenomics is largely related to practical limitations of currently available data that often lack complete characterization of the patients and poor integration of individual datasets. Radiogenomics offers a practical way to leverage limited and incomplete data to generate knowledge that might lead to improved decision making, and as a result, improved patient outcomes.

Computational approach to radiogenomics of breast cancer: Luminal A and luminal B molecular subtypes are associated with imaging features on routine breast MRI extracted using computer vision algorithms.

To identify associations between semiautomatically extracted MRI features and breast cancer molecular subtypes.

Decision optimization of case-based computer-aided decision systems using genetic algorithms with application to mammography

This paper presents an optimization framework for improving case-based computer-aided decision (CB-CAD) systems. The underlying hypothesis of the study is that each example in the knowledge database of a medical decision support system has different importance in the decision making process. A new decision algorithm incorporating an importance weight for each example is proposed to account for these differences. The search for the best set of importance weights is defined as an optimization problem and a genetic algorithm is employed to solve it. The optimization process is tailored to maximize the systems performance according to clinically relevant evaluation criteria. The study was performed using a CAD system developed for the classification of regions of interests (ROIs) in mammograms as depicting masses or normal tissue. The system was constructed and evaluated using a dataset of ROIs extracted from the Digital Database for Screening Mammography (DDSM). Experimental results show that, according to receiver operator characteristic (ROC) analysis, the proposed method significantly improves the overall performance of the CAD system as well as its average specificity for high breast mass detection rates.

Estimating confidence of individual rating predictions in collaborative filtering recommender systems

Collaborative filtering algorithms predict a rating for an item based on the users previous ratings for other items as well as ratings of other users. This approach has gained notable popularity both in academic research and in commercial applications. One aspect of collaborative filtering systems that received interest, but little systematic analysis, is confidence of the rating predictions by collaborative filtering algorithms. In this paper, I address this issue. Specifically: (1) I offer a discussion on the definition of confidence, (2) I propose a method for evaluating performance of confidence estimation algorithms, and (3) I evaluate six different confidence estimation algorithms. Three of those algorithms are introduced in this paper and three have been previously suggested for this purpose. The comparative experimental evaluation demonstrates that two of the algorithms proposed in this study: one using resampling of available ratings and one using noise injection to the available ratings provide the best performance in terms of separation between predictions of high and low confidence. The algorithms that use only the number of ratings available for the user of interest or for the item of interest turned out to be of limited use for confidence estimation.

The effect of class imbalance on case selection for case-based classifiers: An empirical study in the context of medical decision support

Case selection is a useful approach for increasing the efficiency and performance of case-based classifiers. Multiple techniques have been designed to perform case selection. This paper empirically investigates how class imbalance in the available set of training cases can impact the performance of the resulting classifier as well as properties of the selected set. In this study, the experiments are performed using a dataset for the problem of detecting breast masses in screening mammograms. The classification problem was binary and we used a k-nearest neighbor classifier. The classifiers performance was evaluated using the receiver operating characteristic (ROC) area under the curve (AUC) measure. The experimental results indicate that although class imbalance reduces the performance of the derived classifier and the effectiveness of selection at improving overall classifier performance, case selection can still be beneficial, regardless of the level of class imbalance.

Selection of examples in case-based computer-aided decision systems

Case-based computer-aided decision (CB-CAD) systems rely on a database of previously stored, known examples when classifying new, incoming queries. Such systems can be particularly useful since they do not need retraining every time a new example is deposited in the case base. The adaptive nature of case-based systems is well suited to the current trend of continuously expanding digital databases in the medical domain. To maintain efficiency, however, such systems need sophisticated strategies to effectively manage the available evidence database. In this paper, we discuss the general problem of building an evidence database by selecting the most useful examples to store while satisfying existing storage requirements. We evaluate three intelligent techniques for this purpose: genetic algorithm-based selection, greedy selection and random mutation hill climbing. These techniques are compared to a random selection strategy used as the baseline. The study is performed with a previously presented CB-CAD system applied for false positive reduction in screening mammograms. The experimental evaluation shows that when the development goal is to maximize the systems diagnostic performance, the intelligent techniques are able to reduce the size of the evidence database to 37% of the original database by eliminating superfluous and/or detrimental examples while at the same time significantly improving the CAD systems performance. Furthermore, if the case-base size is a main concern, the total number of examples stored in the system can be reduced to only 2-4% of the original database without a decrease in the diagnostic performance. Comparison of the techniques shows that random mutation hill climbing provides the best balance between the diagnostic performance and computational efficiency when building the evidence database of the CB-CAD system.

A fully automatic extraction of magnetic resonance image features in glioblastoma patients.

PURPOSE Glioblastoma is the most common malignant brain tumor. It is characterized by low median survival time and high survival variability. Survival prognosis for glioblastoma is very important for optimized treatment planning. Imaging features observed in magnetic resonance (MR) images were shown to be a good predictor of survival. However, manual assessment of MR features is time-consuming and can be associated with a high inter-reader variability as well as inaccuracies in the assessment. In response to this limitation, the authors proposed and evaluated a computer algorithm that extracts important MR image features in a fully automatic manner. METHODS The algorithm first automatically segmented the available volumes into a background region and four tumor regions. Then, it extracted ten features from the segmented MR imaging volumes, some of which were previously indicated as predictive of clinical outcomes. To evaluate the algorithm, the authors compared the extracted features for 73 glioblastoma patients to the reference standard established by manual segmentation of the tumors. RESULTS The experiments showed that their algorithm was able to extract most of the image features with moderate to high accuracy. High correlation coefficients between the automatically extracted value and reference standard were observed for the tumor location, minor and major axis length as well as tumor volume. Moderately high correlation coefficients were also observed for proportion of enhancing tumor, proportion of necrosis, and thickness of enhancing margin. The correlation coefficients for all these features were statistically significant (p < 0.0001). CONCLUSIONS The authors proposed and evaluated an algorithm that, given a set of MR volumes of a glioblastoma patient, is able to extract MR image features that correlate well with their reference standard. Future studies will evaluate how well the computer-extracted features predict survival.