Justin S. Kirby

The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients - manually annotated by up to four raters - and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans

PURPOSEnThe development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process.nnnMETHODSnSeven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories (nodule > or =3 mm, nodule <3 mm, and non-nodule > or =3 mm). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus.nnnRESULTSnThe Database contains 7371 lesions marked nodule by at least one radiologist. 2669 of these lesions were marked nodule > or =3 mm by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings.nnnCONCLUSIONSnThe LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice.

The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository

The National Institutes of Health have placed significant emphasis on sharing of research data to support secondary research. Investigators have been encouraged to publish their clinical and imaging data as part of fulfilling their grant obligations. Realizing it was not sufficient to merely ask investigators to publish their collection of imaging and clinical data, the National Cancer Institute (NCI) created the open source National Biomedical Image Archive software package as a mechanism for centralized hosting of cancer related imaging. NCI has contracted with Washington University in Saint Louis to create The Cancer Imaging Archive (TCIA)—an open-source, open-access information resource to support research, development, and educational initiatives utilizing advanced medical imaging of cancer. In its first year of operation, TCIA accumulated 23 collections (3.3 million images). Operating and maintaining a high-availability image archive is a complex challenge involving varied archive-specific resources and driven by the needs of both image submitters and image consumers. Quality archives of any type (traditional library, PubMed, refereed journals) require management and customer service. This paper describes the management tasks and user support model for TCIA.

The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI)

PURPOSEnThe development of computer-aided diagnostic (CAD) methods for lung nodule detection, classification, and quantitative assessment can be facilitated through a well-characterized repository of computed tomography (CT) scans. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI) completed such a database, establishing a publicly available reference for the medical imaging research community. Initiated by the National Cancer Institute (NCI), further advanced by the Foundation for the National Institutes of Health (FNIH), and accompanied by the Food and Drug Administration (FDA) through active participation, this public-private partnership demonstrates the success of a consortium founded on a consensus-based process.nnnMETHODSnSeven academic centers and eight medical imaging companies collaborated to identify, address, and resolve challenging organizational, technical, and clinical issues to provide a solid foundation for a robust database. The LIDC/IDRI Database contains 1018 cases, each of which includes images from a clinical thoracic CT scan and an associated XML file that records the results of a two-phase image annotation process performed by four experienced thoracic radiologists. In the initial blinded-read phase, each radiologist independently reviewed each CT scan and marked lesions belonging to one of three categories (nodule > or =3 mm, nodule <3 mm, and non-nodule > or =3 mm). In the subsequent unblinded-read phase, each radiologist independently reviewed their own marks along with the anonymized marks of the three other radiologists to render a final opinion. The goal of this process was to identify as completely as possible all lung nodules in each CT scan without requiring forced consensus.nnnRESULTSnThe Database contains 7371 lesions marked nodule by at least one radiologist. 2669 of these lesions were marked nodule > or =3 mm by at least one radiologist, of which 928 (34.7%) received such marks from all four radiologists. These 2669 lesions include nodule outlines and subjective nodule characteristic ratings.nnnCONCLUSIONSnThe LIDC/IDRI Database is expected to provide an essential medical imaging research resource to spur CAD development, validation, and dissemination in clinical practice.

MR Imaging Predictors of Molecular Profile and Survival: Multi-institutional Study of the TCGA Glioblastoma Data Set

PURPOSEnTo conduct a comprehensive analysis of radiologist-made assessments of glioblastoma (GBM) tumor size and composition by using a community-developed controlled terminology of magnetic resonance (MR) imaging visual features as they relate to genetic alterations, gene expression class, and patient survival.nnnMATERIALS AND METHODSnBecause all study patients had been previously deidentified by the Cancer Genome Atlas (TCGA), a publicly available data set that contains no linkage to patient identifiers and that is HIPAA compliant, no institutional review board approval was required. Presurgical MR images of 75 patients with GBM with genetic data in the TCGA portal were rated by three neuroradiologists for size, location, and tumor morphology by using a standardized feature set. Interrater agreements were analyzed by using the Krippendorff α statistic and intraclass correlation coefficient. Associations between survival, tumor size, and morphology were determined by using multivariate Cox regression models; associations between imaging features and genomics were studied by using the Fisher exact test.nnnRESULTSnInterrater analysis showed significant agreement in terms of contrast material enhancement, nonenhancement, necrosis, edema, and size variables. Contrast-enhanced tumor volume and longest axis length of tumor were strongly associated with poor survival (respectively, hazard ratio: 8.84, P = .0253, and hazard ratio: 1.02, P = .00973), even after adjusting for Karnofsky performance score (P = .0208). Proneural class GBM had significantly lower levels of contrast enhancement (P = .02) than other subtypes, while mesenchymal GBM showed lower levels of nonenhanced tumor (P < .01).nnnCONCLUSIONnThis analysis demonstrates a method for consistent image feature annotation capable of reproducibly characterizing brain tumors; this study shows that radiologists estimations of macroscopic imaging features can be combined with genetic alterations and gene expression subtypes to provide deeper insight to the underlying biologic properties of GBM subsets.

Genomic Mapping and Survival Prediction in Glioblastoma: Molecular Subclassification Strengthened by Hemodynamic Imaging Biomarkers

PURPOSEnTo correlate tumor blood volume, measured by using dynamic susceptibility contrast material-enhanced T2*-weighted magnetic resonance (MR) perfusion studies, with patient survival and determine its association with molecular subclasses of glioblastoma (GBM).nnnMATERIALS AND METHODSnThis HIPAA-compliant retrospective study was approved by institutional review board. Fifty patients underwent dynamic susceptibility contrast-enhanced T2*-weighted MR perfusion studies and had gene expression data available from the Cancer Genome Atlas. Relative cerebral blood volume (rCBV) (maximum rCBV [rCBV(max)] and mean rCBV [rCBV(mean)]) of the contrast-enhanced lesion as well as rCBV of the nonenhanced lesion (rCBV(NEL)) were measured. Patients were subclassified according to the Verhaak and Phillips classification schemas, which are based on similarity to defined genomic expression signature. We correlated rCBV measures with the molecular subclasses as well as with patient overall survival by using Cox regression analysis.nnnRESULTSnNo statistically significant differences were noted for rCBV(max), rCBV(mean) of contrast-enhanced lesion or rCBV(NEL) between the four Verhaak classes or the three Phillips classes. However, increased rCBV measures are associated with poor overall survival in GBM. The rCBV(max) (P = .0131) is the strongest predictor of overall survival regardless of potential confounders or molecular classification. Interestingly, including the Verhaak molecular GBM classification in the survival model clarifies the association of rCBV(mean) with patient overall survival (hazard ratio: 1.46, P = .0212) compared with rCBV(mean) alone (hazard ratio: 1.25, P = .1918). Phillips subclasses are not predictive of overall survival nor do they affect the predictive ability of rCBV measures on overall survival.nnnCONCLUSIONnThe rCBV(max) measurements could be used to predict patient overall survival independent of the molecular subclasses of GBM; however, Verhaak classifiers provided additional information, suggesting that molecular markers could be used in combination with hemodynamic imaging biomarkers in the future.

Outcome Prediction in Patients with Glioblastoma by Using Imaging, Clinical, and Genomic Biomarkers: Focus on the Nonenhancing Component of the Tumor

PURPOSEnTo correlate patient survival with morphologic imaging features and hemodynamic parameters obtained from the nonenhancing region (NER) of glioblastoma (GBM), along with clinical and genomic markers.nnnMATERIALS AND METHODSnAn institutional review board waiver was obtained for this HIPAA-compliant retrospective study. Forty-five patients with GBM underwent baseline imaging with contrast material-enhanced magnetic resonance (MR) imaging and dynamic susceptibility contrast-enhanced T2*-weighted perfusion MR imaging. Molecular and clinical predictors of survival were obtained. Single and multivariable models of overall survival (OS) and progression-free survival (PFS) were explored with Kaplan-Meier estimates, Cox regression, and random survival forests.nnnRESULTSnWorsening OS (log-rank test, P = .0103) and PFS (log-rank test, P = .0223) were associated with increasing relative cerebral blood volume of NER (rCBVNER), which was higher with deep white matter involvement (t test, P = .0482) and poor NER margin definition (t test, P = .0147). NER crossing the midline was the only morphologic feature of NER associated with poor survival (log-rank test, P = .0125). Preoperative Karnofsky performance score (KPS) and resection extent (n = 30) were clinically significant OS predictors (log-rank test, P = .0176 and P = .0038, respectively). No genomic alterations were associated with survival, except patients with high rCBVNER and wild-type epidermal growth factor receptor (EGFR) mutation had significantly poor survival (log-rank test, P = .0306; area under the receiver operating characteristic curve = 0.62). Combining resection extent with rCBVNER marginally improved prognostic ability (permutation, P = .084). Random forest models of presurgical predictors indicated rCBVNER as the top predictor; also important were KPS, age at diagnosis, and NER crossing the midline. A multivariable model containing rCBVNER, age at diagnosis, and KPS can be used to group patients with more than 1 year of difference in observed median survival (0.49-1.79 years).nnnCONCLUSIONnPatients with high rCBVNER and NER crossing the midline and those with high rCBVNER and wild-type EGFR mutation showed poor survival. In multivariable survival models, however, rCBVNER provided unique prognostic information that went above and beyond the assessment of all NER imaging features, as well as clinical and genomic features.

Image data sharing for biomedical research--meeting HIPAA requirements for De-identification.

Data sharing is increasingly recognized as critical to cross-disciplinary research and to assuring scientific validity. Despite National Institutes of Health and National Science Foundation policies encouraging data sharing by grantees, little data sharing of clinical data has in fact occurred. A principal reason often given is the potential of inadvertent violation of the Health Insurance Portability and Accountability Act privacy regulations. While regulations specify the components of private health information that should be protected, there are no commonly accepted methods to de-identify clinical data objects such as images. This leads institutions to take conservative risk-averse positions on data sharing. In imaging trials, where images are coded according to the Digital Imaging and Communications in Medicine (DICOM) standard, the complexity of the data objects and the flexibility of the DICOM standard have made it especially difficult to meet privacy protection objectives. The recent release of DICOM Supplement 142 on image de-identification has removed much of this impediment. This article describes the development of an open-source software suite that implements DICOM Supplement 142 as part of the National Biomedical Imaging Archive (NBIA). It also describes the lessons learned by the authors as NBIA has acquired more than 20 image collections encompassing over 30 million images.

Addition of MR imaging features and genetic biomarkers strengthens glioblastoma survival prediction in TCGA patients

PURPOSEnThe purpose of our study was to assess whether a model combining clinical factors, MR imaging features, and genomics would better predict overall survival of patients with glioblastoma (GBM) than either individual data type.nnnMETHODSnThe study was conducted leveraging The Cancer Genome Atlas (TCGA) effort supported by the National Institutes of Health. Six neuroradiologists reviewed MRI images from The Cancer Imaging Archive (http://cancerimagingarchive.net) of 102 GBM patients using the VASARI scoring system. The patients clinical and genetic data were obtained from the TCGA website (http://www.cancergenome.nih.gov/). Patient outcome was measured in terms of overall survival time. The association between different categories of biomarkers and survival was evaluated using Cox analysis.nnnRESULTSnThe features that were significantly associated with survival were: (1) clinical factors: chemotherapy; (2) imaging: proportion of tumor contrast enhancement on MRI; and (3) genomics: HRAS copy number variation. The combination of these three biomarkers resulted in an incremental increase in the strength of prediction of survival, with the model that included clinical, imaging, and genetic variables having the highest predictive accuracy (area under the curve 0.679±0.068, Akaikes information criterion 566.7, P<0.001).nnnCONCLUSIONnA combination of clinical factors, imaging features, and HRAS copy number variation best predicts survival of patients with GBM.

Radiogenomics of clear cell renal cell carcinoma: preliminary findings of The Cancer Genome Atlas–Renal Cell Carcinoma (TCGA–RCC) Imaging Research Group

PurposeTo investigate associations between imaging features and mutational status of clear cell renal cell carcinoma (ccRCC).Materials and methodsThis multi-institutional, multi-reader study included 103 patients (77 men; median age 59xa0years, range 34–79) with ccRCC examined with CT in 81 patients, MRI in 19, and both CT and MRI in three; images were downloaded from The Cancer Imaging Archive, an NCI-funded project for genome-mapping and analyses. Imaging features [size (mm), margin (well-defined or ill-defined), composition (solid or cystic), necrosis (for solid tumors: 0%, 1%–33%, 34%–66% or >66%), growth pattern (endophytic, <50% exophytic, or ≥50% exophytic), and calcification (present, absent, or indeterminate)] were reviewed independently by three readers blinded to mutational data. The association of imaging features with mutational status (VHL, BAP1, PBRM1, SETD2, KDM5C, and MUC4) was assessed.ResultsMedian tumor size was 49xa0mm (range 14–162xa0mm), 73 (71%) tumors had well-defined margins, 98 (95%) tumors were solid, 95 (92%) showed presence of necrosis, 46 (45%) had ≥50% exophytic component, and 18 (19.8%) had calcification. VHL (nxa0=xa052) and PBRM1 (nxa0=xa024) were the most common mutations. BAP1 mutation was associated with ill-defined margin and presence of calcification (pxa0=xa00.02 and 0.002, respectively, Pearson’s χ2 test); MUC4 mutation was associated with an exophytic growth pattern (pxa0=xa00.002, Mann–Whitney U test).ConclusionsBAP1 mutation was associated with ill-defined tumor margins and presence of calcification; MUC4 mutation was associated with exophytic growth. Given the known prognostic implications of BAP1 and MUC4 mutations, these results support using radiogenomics to aid in prognostication and management.