Marios A. Gavrielides

Observer Variability in the Interpretation of HER2/neu Immunohistochemical Expression With Unaided and Computer-Aided Digital Microscopy

CONTEXT Observer variability in digital microscopy and the effect of computer-aided digital microscopy are underexamined areas in need of further research, considering the increasing use and future role of digital imaging in pathology. A reduction in observer variability using computer aids could enhance the statistical power of studies designed to determine the utility of new biomarkers and accelerate their incorporation in clinical practice. OBJECTIVES To quantify interobserver and intraobserver variability in immunohistochemical analysis of HER2/neu with digital microscopy and computer-aided digital microscopy, and to test the hypothesis that observer agreement in the quantitative assessment of HER2/neu immunohistochemical expression is increased with the use of computer-aided microscopy. DESIGN A set of 335 digital microscopy images extracted from 64 breast cancer tissue slides stained with a HER2 antibody, were read by 14 observers in 2 reading modes: the unaided mode and the computer-aided mode. In the unaided mode, HER2 images were displayed on a calibrated color monitor with no other information, whereas in the computer-aided mode, observers were shown a HER2 image along with a corresponding feature plot showing computer-extracted values of membrane staining intensity and membrane completeness for the particular image under examination and, at the same time, mean feature values of the different HER2 categories. In both modes, observers were asked to provide a continuous score of HER2 expression. RESULTS Agreement analysis performed on the output of the study showed significant improvement in both interobserver and intraobserver agreement when the computer-aided reading mode was used to evaluate preselected image fields. CONCLUSION The role of computer-aided digital microscopy in reducing observer variability in immunohistochemistry is promising.

A resource for the assessment of lung nodule size estimation methods: database of thoracic CT scans of an anthropomorphic phantom

A number of interrelated factors can affect the precision and accuracy of lung nodule size estimation. To quantify the effect of these factors, we have been conducting phantom CT studies using an anthropomorphic thoracic phantom containing a vasculature insert to which synthetic nodules were inserted or attached. Ten repeat scans were acquired on different multi-detector scanners, using several sets of acquisition and reconstruction protocols and various nodule characteristics (size, shape, density, location). This study design enables both bias and variance analysis for the nodule size estimation task. The resulting database is in the process of becoming publicly available as a resource to facilitate the assessment of lung nodule size estimation methodologies and to enable comparisons between different methods regarding measurement error. This resource complements public databases of clinical data and will contribute towards the development of procedures that will maximize the utility of CT imaging for lung cancer screening and tumor therapy evaluation.

Benefit of Overlapping Reconstruction for Improving the Quantitative Assessment of CT Lung Nodule Volume

RATIONALE AND OBJECTIVES The aim of this study was to quantify the effect of overlapping reconstruction on the precision and accuracy of lung nodule volume estimates in a phantom computed tomographic (CT) study. MATERIALS AND METHODS An anthropomorphic phantom was used with a vasculature insert on which synthetic lung nodules were attached. Repeated scans of the phantom were acquired using a 64-slice CT scanner. Overlapping and contiguous reconstructions were performed for a range of CT imaging parameters (exposure, slice thickness, pitch, reconstruction kernel) and a range of nodule characteristics (size, density). Nodule volume was estimated with a previously developed matched-filter algorithm. RESULTS Absolute percentage bias across all nodule sizes (n = 2880) was significantly lower when overlapping reconstruction was used, with an absolute percentage bias of 6.6% (95% confidence interval [CI], 6.4-6.9), compared to 13.2% (95% CI, 12.7-13.8) for contiguous reconstruction. Overlapping reconstruction also showed a precision benefit, with a lower standard percentage error of 7.1% (95% CI, 6.9-7.2) compared with 15.3% (95% CI, 14.9-15.7) for contiguous reconstructions across all nodules. Both effects were more pronounced for the smaller, subcentimeter nodules. CONCLUSIONS These results support the use of overlapping reconstruction to improve the quantitative assessment of nodule size with CT imaging.

Reproducibility in the automated quantitative assessment of HER2/neu for breast cancer

Background: With the emerging role of digital imaging in pathology and the application of automated image-based algorithms to a number of quantitative tasks, there is a need to examine factors that may affect the reproducibility of results. These factors include the imaging properties of whole slide imaging (WSI) systems and their effect on the performance of quantitative tools. This manuscript examines inter-scanner and inter-algorithm variability in the assessment of the commonly used HER2/neu tissue-based biomarker for breast cancer with emphasis on the effect of algorithm training. Materials and Methods: A total of 241 regions of interest from 64 breast cancer tissue glass slides were scanned using three different whole-slide images and were analyzed using two different automated image analysis algorithms, one with preset parameters and another incorporating a procedure for objective parameter optimization. Ground truth from a panel of seven pathologists was available from a previous study. Agreement analysis was used to compare the resulting HER2/neu scores. Results: The results of our study showed that inter-scanner agreement in the assessment of HER2/neu for breast cancer in selected fields of view when analyzed with any of the two algorithms examined in this study was equal or better than the inter-observer agreement previously reported on the same set of data. Results also showed that discrepancies observed between algorithm results on data from different scanners were significantly reduced when the alternative algorithm that incorporated an objective re-training procedure was used, compared to the commercial algorithm with preset parameters. Conclusion: Our study supports the use of objective procedures for algorithm training to account for differences in image properties between WSI systems.

Approximations of noise covariance in multi-slice helical CT scans: impact on lung nodule size estimation

Multi-slice computed tomography (MSCT) scanners have become popular volumetric imaging tools. Deterministic and random properties of the resulting CT scans have been studied in the literature. Due to the large number of voxels in the three-dimensional (3D) volumetric dataset, full characterization of the noise covariance in MSCT scans is difficult to tackle. However, as usage of such datasets for quantitative disease diagnosis grows, so does the importance of understanding the noise properties because of their effect on the accuracy of the clinical outcome. The goal of this work is to study noise covariance in the helical MSCT volumetric dataset. We explore possible approximations to the noise covariance matrix with reduced degrees of freedom, including voxel-based variance, one-dimensional (1D) correlation, two-dimensional (2D) in-plane correlation and the noise power spectrum (NPS). We further examine the effect of various noise covariance models on the accuracy of a prewhitening matched filter nodule size estimation strategy. Our simulation results suggest that the 1D longitudinal, 2D in-plane and NPS prewhitening approaches can improve the performance of nodule size estimation algorithms. When taking into account computational costs in determining noise characterizations, the NPS model may be the most efficient approximation to the MSCT noise covariance matrix.

Statistical analysis of lung nodule volume measurements with CT in a large‐scale phantom study

PURPOSE To determine inter-related factors that contribute substantially to measurement error of pulmonary nodule measurements with CT by assessing a large-scale dataset of phantom scans and to quantitatively validate the repeatability and reproducibility of a subset containing nodules and CT acquisitions consistent with the Quantitative Imaging Biomarker Alliance (QIBA) metrology recommendations. METHODS The dataset has about 40 000 volume measurements of 48 nodules (5-20 mm, four shapes, three radiodensities) estimated by a matched-filter estimator from CT images involving 72 imaging protocols. Technical assessment was performed under a framework suggested by QIBA, which aimed to minimize the inconsistency of terminologies and techniques used in the literature. Accuracy and precision of lung nodule volume measurements were examined by analyzing the linearity, bias, variance, root mean square error (RMSE), repeatability, reproducibility, and significant and substantial factors that contribute to the measurement error. Statistical methodologies including linear regression, analysis of variance, and restricted maximum likelihood were applied to estimate the aforementioned metrics. The analysis was performed on both the whole dataset and a subset meeting the criteria proposed in the QIBA Profile document. RESULTS Strong linearity was observed for all data. Size, slice thickness × collimation, and randomness in attachment to vessels or chest wall were the main sources of measurement error. Grouping the data by nodule size and slice thickness × collimation, the standard deviation (3.9%-28%), and RMSE (4.4%-68%) tended to increase with smaller nodule size and larger slice thickness. For 5, 8, 10, and 20 mm nodules with reconstruction slice thickness ≤0.8, 3, 3, and 5 mm, respectively, the measurements were almost unbiased (-3.0% to 3.0%). Repeatability coefficients (RCs) were from 6.2% to 40%. Pitch of 0.9, detail kernel, and smaller slice thicknesses yielded better (smaller) RCs than those from pitch of 1.2, medium kernel, and larger slice thicknesses. Exposure showed no impact on RC. The overall reproducibility coefficient (RDC) was 45%, and reduced to about 20%-30% when the slice thickness and collimation were fixed. For nodules and CT imaging complying with the QIBA Profile (QIBA Profile subset), the measurements were highly repeatable and reproducible in spite of variations in nodule characteristics and imaging protocols. The overall measurement error was small and mostly due to the randomness in attachment. The bias, standard deviation, and RMSE grouped by nodule size and slice thickness × collimation in the QIBA Profile subset were within ±3%, 4%, and 5%, respectively. RCs are within 11% and the overall RDC is equal to 11%. CONCLUSIONS The authors have performed a comprehensive technical assessment of lung nodule volumetry with a matched-filter estimator from CT scans of synthetic nodules and identified the main sources of measurement error among various nodule characteristics and imaging parameters. The results confirm that the QIBA Profile set is highly repeatable and reproducible. These phantom study results can serve as a bound on the clinical performance achievable with volumetric CT measurements of pulmonary nodules.

Pulmonary nodules with ground-glass opacity can be reliably measured with low-dose techniques regardless of iterative reconstruction: results of a phantom study.

OBJECTIVE Pulmonary nodules of ground-glass opacity represent one imaging manifestation of a slow-growing variant of lung cancer. The objective of this phantom study was to quantify the effect of the radiation dose used for the examination (volume CT dose index [CTDI(vol)]), type of reconstruction algorithm, and choice of postreconstruction enhancement algorithms on the measurement error when assessing the volume of simulated lung nodules with CT, focusing on two radiodensity levels. MATERIALS AND METHODS Twelve synthetic nodules of two radiodensities (-630 and -10 HU), three shapes (spherical, lobulated, and spiculated), and two sizes (nominal diameters of 5 and 10 mm) were inserted into an anthropomorphic chest phantom and scanned with techniques varying in CTDI(vol) (from subscreening dose [0.8 mGy] to diagnostic levels [6.5 mGy]), reconstruction algorithms (iterative reconstruction and filtered back projection), and different postreconstruction enhancement algorithms. Nodule volume was measured from the resulting reconstructed CT images with a matched filter estimator. RESULTS No significant over- or underestimation of nodule volume was observed across individual variables, with low percentage error overall (-1.4%) and for individual variables (range, -3.4% to 0.4%). The magnitude of percentage error was also low (overall average percentage error < 6% and SD values < 4.5%) and for individual variables (absolute percentage error range 3.3-5.6%). No clinically significant differences were observed between different levels of CTDI(vol), use of iterative reconstruction algorithms, or use of different postreconstruction enhancement algorithms. CONCLUSION These results indicate that, if validated for other measurement tools and scanners, lung nodule volume measurements from scans acquired and reconstructed with significantly different acquisition and reconstruction techniques can be reliably compared.

Assessing color reproducibility of whole-slide imaging scanners

A new method for assessing color reproducibility of whole-slide imaging (WSI) systems is introduced. A color phantom is used to evaluate the difference between the input to and the output from a WSI system. The method consists of four components: (a) producing the color phantom, (b) establishing the truth of the color phantom, (c) retrieving the digital display data from the WSI system, and (d) calculating the color difference. The method was applied to a WSI system and used to evaluate the color characteristics with and without color management.

Quantitative Assessment and Classification of Tissue-Based Biomarker Expression With Color Content Analysis

CONTEXT The use of computer aids has been suggested as a way to reduce interobserver variability that is known to exist in the interpretation of immunohistochemical staining in pathology. Such computer aids should be automated in their usage but also they should be trained in an automated and reproducible fashion. OBJECTIVE To present a computer aid for the quantitative analysis of tissue-based biomarkers, based on color content analysis. DESIGN The developed system incorporates an automated algorithm to allow retraining based on the color properties of different training sets. The algorithm first generates a color palette containing the colors present in a training subset. Based on the palette, color histograms are derived and are used as feature vectors to a pattern recognition system, which returns an output proportional to biomarker continuous expression or a categorical classification. The method was evaluated on a database of HER2/neu digital breast cancer slides, for which expression scores from a pathologist panel were available. The system was retrained and evaluated on different transformations of the database, including compression, blurring, and changes in illumination, to examine its robustness to different imaging conditions frequently met in digital pathology. RESULTS Results showed high agreement between the results of the algorithm and the truth from the pathologist panel as well as robustness to image transformations. CONCLUSIONS The results of the study are encouraging for the potential of this method as a computer aid to assess biomarker expression in a consistent and reproducible manner.

Evaluation of 1D, 2D and 3D nodule size estimation by radiologists for spherical and non-spherical nodules through CT thoracic phantom imaging

The purpose of this work was to estimate bias in measuring the size of spherical and non-spherical lesions by radiologists using three sizing techniques under a variety of simulated lesion and reconstruction slice thickness conditions. We designed a reader study in which six radiologists estimated the size of 10 synthetic nodules of various sizes, shapes and densities embedded within a realistic anthropomorphic thorax phantom from CT scan data. In this manuscript we report preliminary results for the first four readers (Reader 1-4). Two repeat CT scans of the phantom containing each nodule were acquired using a Philips 16-slice scanner at a 0.8 and 5 mm slice thickness. The readers measured the sizes of all nodules for each of the 40 resulting scans (10 nodules x 2 slice thickness x 2 repeat scans) using three sizing techniques (1D longest in-slice dimension; 2D area from longest in-slice dimension and corresponding longest perpendicular dimension; 3D semi-automated volume) in each of 2 reading sessions. The normalized size was estimated for each sizing method and an inter-comparison of bias among methods was performed. The overall relative biases (standard deviation) of the 1D, 2D and 3D methods for the four readers subset (Readers 1-4) were -13.4 (20.3), -15.3 (28.4) and 4.8 (21.2) percentage points, respectively. The relative biases for the 3D volume sizing method was statistically lower than either the 1D or 2D method (p<0.001 for 1D vs. 3D and 2D vs. 3D).