Cynthia A. Britton

Empiric assessment of parameters that affect the design of multireader receiver operating characteristic studies

RATIONALE AND OBJECTIVES The authors attempted to assess experimentally the magnitude of reader variability and the correlations and interactions among cases, readers, and modalities during observer performance studies and their possible effects on study design and sample size. MATERIALS AND METHODS Published data from 32 selected receiver operating characteristic (ROC) studies were reviewed to compare the magnitude of the variance component from readers with the variance component from modality. Estimates of correlation and interactions among cases, readers, and modalities were also computed directly from ROC data ascertained during two large studies performed in our laboratory. Each of these two studies included 529 cases and six readers, but one study used eight modalities and the other nine. RESULTS Published results indicate that reader variability is task dependent and larger (P < .05) than modality variability in detection of interstitial disease. Measured correlations between modalities for the same reader were task dependent and ranged from 0.35 to 0.59. Modality-by-reader and modality-by-case interactions often are not important factors. The random error term was greater than the modality-by-reader interaction in 11 of 20 comparisons and greater than the modality-by-case interaction in eight of 20 comparisons. CONCLUSION Use of the same cases interpreted with different modes is justifiable in many situations because of the high variability from readers. This comprehensive review of existing ROC studies resulted in parameter assessments that can be used to better estimate sample-size requirements in multireader ROC studies.

The pronator quadratus: A distinct forearm space?*

The pronator quadratus sign is associated with fractures of the distal radius and ulna and is believed to be due to accumulation of fluid within the pronator quadratus muscle. This anatomic study based on dye injection and x-ray film examination shows that the pronator quadratus occupies a distinct forearm space without intramuscular communication.

Subjective and objective assessment of image quality—A comparison

Forced-choice just noticeable difference (JND) studies are extremely sensitive to image quality variations that are below the threshold at which the differences are apparent to or definable by the observer. Paired comparisons of 4K and 2K laser-printed posteroanterior chest images consistently demonstrated that although images are viewed as comparable by radiologists, when forced to choose the better (“sharper”) image, they actually select the higher-resolution images in 83% of the paired observations. We conclude that small differences in image quality may be detectable even in image sets which are considered to be comparable by subjective assessments.

Practical Issues of Experimental ROC Analysis Selection of Controls

Receiver operating characteristics (ROC) analysis has been used in many medical imaging applications during the past decade. With the recent expansion of the ROC methodology to multi-disease studies, several reports have begun to address both the theoretical and experimental design issues associated with such studies. While the appropriate selection, classification, and verification of actually positive cases is carefully addressed in the literature, similar considerations are rarely given to the selection of actually negative controls for these studies. In this paper, theoretical considerations and experimental data are provided to demonstrate the significance of this very issue.

Vertebral fractures and trabecular microstructure in men with prostate cancer on androgen deprivation therapy

Androgen deprivation therapy (ADT), a treatment for prostate cancer, is associated with bone loss and fractures. Dual‐energy X‐ray absorptiometry (DXA)–measured bone mineral density does not assess vertebral fractures (VF). High‐resolution micro‐magnetic resonance imaging (HR‐MRI) assesses bone microarchitecture and provides structural information. To determine if VF identification increased the diagnosis of osteoporosis beyond DXA and if HR‐MRI demonstrated skeletal deterioration in men with VF, we cross‐sectionally studied 137 men aged ≥ 60 years with nonmetastatic prostate cancer on ADT for ≥ 6 months. Vertebral fracture assessment (VFA) by DXA was confirmed with X‐rays. HR‐MRI of the wrist included bone volume to total volume (BV/TV), surface density (trabecular plates), surface/curve ratio (plates/rods), and erosion index (higher depicts deterioration). VF were found in 37% of men; the majority were unknown. Seven percent of participants were classified as osteoporotic by hip or spine DXA. Thirty‐seven percent of men without osteoporosis by DXA had VF identified, suggesting that 90% of patients with clinical osteoporosis would have been misclassified by DXA alone. By ANOVA comparison across VF grades, the BV/TV, surface density, and spine, hip, and wrist DXA were lower, and erosion index was higher in men with moderate‐severe VF compared with lesser grades (all p < 0.05). By unadjusted ROC analysis, the addition of HR‐MRI to DXA at the spine, hip, and femoral neck added substantially (AUC increased 0.831 to 0.902, p < 0.05) to prediction of moderate‐severe vertebral fracture. HR‐MRI indices were associated with spine, hip, and wrist DXA measures (p < 0.01). Longer duration of ADT was associated with lower BV/TV, surface density, and surface/curve ratio (p < 0.05). ADT for men with prostate cancer is associated with silent VF. DXA alone leads to misclassifications of osteoporosis, which can be avoided by VF assessment. HR‐MRI provides a novel technique to assess deterioration of structural integrity in men with VF and adds micro‐structural information.

Characterization of Radiologists’ Search Strategies for Lung Nodule Detection: Slice-Based Versus Volumetric Displays

The goal of this study was to assess whether radiologists’ search paths for lung nodule detection in chest computed tomography (CT) between different rendering and display schemes have reliable properties that can be exploited as an indicator of ergonomic efficiency for the purpose of comparing different display paradigms. Eight radiologists retrospectively viewed 30 lung cancer screening CT exams, containing a total of 91 nodules, in each of three display modes [i.e., slice-by-slice, orthogonal maximum intensity projection (MIP) and stereoscopic] for the purpose of detecting and classifying lung nodules. Radiologists’ search patterns in the axial direction were recorded and analyzed along with the location, size, and shape for each detected feature, and the likelihood that the feature is an actual nodule. Nodule detection performance was analyzed by employing free-response receiver operating characteristic methods. Search paths were clearly different between slice-by-slice displays and volumetric displays but, aside from training and novelty effects, not between MIP and stereographic displays. Novelty and training effects were associated with the stereographic display mode, as evidenced by differences between the beginning and end of the study. The stereo display provided higher detection and classification performance with less interpretation time compared to other display modes tested in the study; however, the differences were not statistically significant. Our preliminary results indicate a potential role for the use of radiologists’ search paths in evaluating the relative ergonomic efficiencies of different display paradigms, but systematic training and practice is necessary to eliminate training curve and novelty effects before search strategies can be meaningfully compared.

Different Search Patterns and Similar Decision Outcomes: How Can Experts Agree in the Decisions They Make When Reading Digital Mammograms?

Experts may agree in most decisions that they make when they read a case set of digital mammograms, but eye-position tracking studies suggest that they use very different visual search strategies to make such decisions. If indeed each expert uses a unique strategy, it may be very difficult to teach radiology trainees effective ways to search the background parenchyma. In this study, we examined how much agreement exists in the actual locations used by the experts in their decision making process when reading digital mammograms.

Selection of processing algorithms for digital image compression: a rank-order study.

RATIONALE AND OBJECTIVES We investigated non-receiver operating characteristic (non-ROC) methods for the selection of processing algorithms for digital image compression. METHODS We performed a multipoint, rank-order study with 20 posteroanterior chest images, each processed using four different algorithms. Seven radiologists reviewed these alongside the digitized noncompressed image. Observers were forced to rank order the similarity and/or difference of the processed images to the nonprocessed image in each case. RESULTS A two-way analysis of variance of the rankings was statistically significant (p = .025), indicating that one processing scheme yielded images that were clearly perceived as the most similar to the nonprocessed images. The selected processing scheme was not the one that yielded the lowest quantitative difference from the nonprocessed images as measured by root mean square error. CONCLUSION Non-ROC study designs that are highly sensitive to small differences among similar images can be used to select processing algorithms.

Projection Models for Stereo Display of Chest CT

The widespread adoption of chest CT for lung cancer screening will greatly increase the workload of chest radiologists. Contributing to this effort is the need for radiologists to differentiate between localized nodules and slices through linear structures such as blood vessels, in each of a large number of slices acquired for each subject. To increase efficiency and accuracy, thin slices can be combined to provide thicker slabs for presentation, but the resulting superposition of tissues can make it more difficult to detect and characterize smaller nodules. The stereo display of a stack of thin CT slices may be able to clarify three-dimensional structures, while avoiding the loss of resolution and ambiguities due to tissue superposition. The current work focuses on the development and evaluation of stereo projection models that are appropriate for chest CT. As slices are combined into a three dimensional structure, maximum image intensity, which is limited by the display, must be preserved. But, compositing methods that effectively average slices together typically reduce contrast of subtle nodules. For monoscopic viewing, orthographic maximum-intensity projection (MIP), of thick slabs, has been employed to overcome this effect, but this method provides no information of depth or of the geometrical relationships between structures. Our comparison of various rendering options indicates that a stereographic perspective transformation, used in conjunction with a compositing model that combines maximum-intensity projection with an appropriate brightness weighting function, shows promise for this application. The main drawback uncovered was that, for the images used in this study, the lung volume was undersampled in the z-direction, resulting in certain unavoidable image artifacts.

Ultra‐high‐field RF coil development for evaluating upper extremity imaging applications

The purpose of this study is to develop and evaluate a custom‐designed 7 T MRI coil and explore its use for upper extremity applications.