Warren Reed

Malignancy Detection in Digital Mammograms: Important Reader Characteristics and Required Case Numbers

RATIONALE AND OBJECTIVES To determine the relationship between heightened levels of reader performance and reader practice in terms of number of cases read and previous experience. MATERIALS AND METHODS A test set of mammograms was developed comprising 50 cases. These cases consisted of 15 abnormals (biopsy proven) and 35 normals (confirmed at subsequent rescreen). Sixty-nine breast image readers reviewed these cases independently and their performance was measured by recording their individual receiver operating characteristic score (area under the curve), sensitivity, and specificity. These measures of performance were then compared to a range of factors relating to the reader such as years of certification and reporting, number of cases read per year, previous experiences, and satisfaction levels. Correlation analyses using Spearman methods were performed along with the Mann-Whitney test to detect differences in performance between specific reader groups. RESULTS Improved reader performance was found for years certified (P = .004), years of experience (P = .0001), and hours reading per week (P = .003) shown by positive statistical significant relationships with Az values (area under receiver operating characteristic curve). Statistical comparisons of Az values scored for individuals who read varying number of cases per year showed that those individuals whose annual mammographic case load was 5000 or more (P = .03) or between 2000 and 4999 (P = .05), had statistically significantly higher scores than those who read less than 1000 cases per year. CONCLUSION The results of this study have shown variations in reader performance relating to parameters of reader practice and experience. Levels of variance are shown and potential acceptance levels for diagnostic efficacy are proposed which may inform policy makers, judicial systems and public debate.

Screening mammography: test set data can reasonably describe actual clinical reporting.

PURPOSE To establish the extent to which test set reading can represent actual clinical reporting in screening mammography. MATERIALS AND METHODS Institutional ethics approval was granted, and informed consent was obtained from each participating screen reader. The need for informed consent with respect to the use of patient materials was waived. Two hundred mammographic examinations were selected from examinations reported by 10 individual expert screen readers, resulting in 10 reader-specific test sets. Data generated from actual clinical reports were compared with three test set conditions: clinical test set reading with prior images, laboratory test set reading with prior images, and laboratory test set reading without prior images. A further set of five expert screen readers was asked to interpret a common set of images in two identical test set conditions to establish a baseline for intraobserver variability. Confidence scores (from 1 to 4) were assigned to the respective decisions made by readers. Region-of-interest (ROI) figures of merit (FOMs) and side-specific sensitivity and specificity were described for the actual clinical reporting of each reader-specific test set and were compared with those for the three test set conditions. Agreement between pairs of readings was performed by using the Kendall coefficient of concordance. RESULTS Moderate or acceptable levels of agreement were evident (W = 0.69-0.73, P < .01) when describing group performance between actual clinical reporting and test set conditions that were reasonably close to the established baseline (W = 0.77, P < .01) and were lowest when prior images were excluded. Higher median values for ROI FOMs were demonstrated for the test set conditions than for the actual clinical reporting values; this was possibly linked to changes in sensitivity. CONCLUSION Reasonable levels of agreement between actual clinical reporting and test set conditions can be achieved, although inflated sensitivity may be evident with test set conditions.

Assessing reader performance in radiology, an imperfect science: lessons from breast screening.

The purpose of this article is to review the limitations associated with current methods of assessing reader accuracy in mammography screening programmes. Clinical audit is commonly used as a quality-assurance tool to monitor the performance of screen readers; however, a number of the metrics employed, such as recall rate as a surrogate for specificity, do not always accurately measure the intended clinical feature. Alternatively, standardized screening test sets, which benefit from ease of application, immediacy of results, and quicker assessment of quality improvement plans, suffer from experimental confounders, thus questioning the relevance of these laboratory-type screening test sets to clinical performance. Four key factors that impact on the external validity of screening test sets were identified: the nature and extent of scrutiny of ones action, the artificiality of the environment, the over-simplification of responses, and prevalence of abnormality. The impact of these factors on radiological and other contexts is discussed, and although it is important to acknowledge the benefit of standardized screening test sets, issues relating to the relevance of test sets to clinical activities remain. The degree of correlation between performance based on real-life clinical audit and performances at screen read test sets must be better understood and specific causal agents for any lack of correlation identified.

Impact of biplane versus single-plane imaging on radiation dose, contrast load and procedural time in coronary angioplasty

Coronary angioplasties can be performed with either single-plane or biplane imaging techniques. The aim of this study was to determine whether biplane imaging, in comparison to single-plane imaging, reduces radiation dose and contrast load and shortens procedural time during (i) primary and elective coronary angioplasty procedures, (ii) angioplasty to the main vascular territories and (iii) procedures performed by operators with various levels of experience. This prospective observational study included a total of 504 primary and elective single-vessel coronary angioplasty procedures utilising either biplane or single-plane imaging. Radiographic and clinical parameters were collected from clinical reports and examination protocols. Radiation dose was measured by a dose-area-product (DAP) meter intrinsic to the angiography system. Our results showed that biplane imaging delivered a significantly greater radiation dose (181.4+/-121.0 Gycm(2)) than single-plane imaging (133.6+/-92.8 Gycm(2), p<0.0001). The difference was independent of case type (primary or elective) (p = 0.862), vascular territory (p = 0.519) and operator experience (p = 0.903). No significant difference was found in contrast load between biplane (166.8+/-62.9 ml) and single-plane imaging (176.8+/-66.0 ml) (p = 0.302). This non-significant difference was independent of case type (p = 0.551), vascular territory (p = 0.308) and operator experience (p = 0.304). Procedures performed with biplane imaging were significantly longer (55.3+/-27.8 min) than those with single-plane (48.9+/-24.2 min, p = 0.010) and, similarly, were not dependent on case type (p = 0.226), vascular territory (p = 0.642) or operator experience (p = 0.094). Biplane imaging resulted in a greater radiation dose and a longer procedural time and delivered a non-significant reduction in contrast load than single-plane imaging. These findings did not support the commonly perceived advantages of using biplane imaging in single-vessel coronary interventional procedures.

Can prevalence expectations drive radiologists' behavior?

RATIONALE AND OBJECTIVES To measure the effect of explicit prevalence expectation on the performance of experienced radiologists during image interpretation of pulmonary lesions on chest radiographs. MATERIALS AND METHODS Each of 22 experienced radiologists was allocated to one of three groups to interpret a set of 30 (15 abnormal) posteroanterior chest images on two occasions to decide if pulmonary lesions were present. Before each viewing, the radiologists were told that the images contained a specific number of abnormal images: group 1, 9 versus 15; group 2, 22 versus 15; and group 3, not told versus 15, respectively. Eye position metrics and receiver operating characteristics confidence ratings were compared for normal and abnormal images. An analysis of false-positive and false-negative decisions was also performed. RESULTS For normal images, at higher prevalence expectation, significant increases were noted for duration of image scrutiny (group 1: P = .0004; group 2: P = .007; and group 3: P = .003) and number of fixations per image (group 1: P = .0006; group 2: P = .0004; and group 3: P = .0001). Also for normal images, group 1 demonstrated a significant increase (P = .038) in average confidence ratings when prevalence expectation increased. For abnormal images, at higher prevalence expectation, significant increases were noted for duration of image scrutiny in group 1 (P = .005) and number of fixations per image in group 1 (P = .01) and group 2 (P = .003). CONCLUSIONS Confidence ratings and visual search of the expert radiologists appear to be affected by changing prevalence expectations. The impact of prevalence expectation appears to be more apparent for normal images.

Reader practice in mammography screen reporting in Australia.

Reader variability is a problem in mammography image reporting and compromises the efficacy of screening programmes. The purpose of this exploratory study was to survey reader practice in reporting screening mammograms in Australia to identify aspects of practice that warrant further investigation. Mammography reporting practice and influences on concentration and attention were investigated by using an original questionnaire distributed to screen readers in Australia. A response rate of 71% (83 out of 117) was achieved. Demographic data indicated that the majority of readers were over 46 years of age (73%), have been reporting on screening mammograms for over 10 years (61%), take less than 1 min to report upon a screening mammogram examination (66%), report up to 200 examinations in a single session (83%) and take up to 2 h to report one session (61%). A majority report on more than 5000 examinations annually (66%); 93% of participants regard their search strategy as systematic, 87% agreed that their concentration can vary throughout a session, 64% agreed that the relatively low number of positives can lead to lapses in concentration and attention and almost all (94%) participants agreed that methods to maximise concentration should be explored. Participants identified a range of influences on concentration within their working environment including volume of images reported in one session, image types and aspects of the physical environment. This study has provided important evidence of the need to investigate adverse influences on concentration during mammography screen reporting

Neuroimaging in adult penetrating brain injury: a guide for radiographers.

Penetrating brain injuries (PBI) are a medical emergency, often resulting in complex damage and high mortality rates. Neuroimaging is essential to evaluate the location and extent of injuries, and to manage them accordingly. Currently, a myriad of imaging modalities are included in the diagnostic workup for adult PBI, including skull radiography, computed tomography (CT), magnetic resonance imaging (MRI) and angiography, with each modality providing their own particular benefits. This literature review explores the current modalities available for investigating PBI and aims to assist in decision making for the appropriate use of diagnostic imaging when presented with an adult PBI. Based on the current literature, the authors have developed an imaging pathway for adult penetrating brain injury that functions as both a learning tool and reference guide for radiographers and other health professionals. Currently, CT is recommended as the imaging modality of choice for the initial assessment of PBI patients, while MRI is important in the sub‐acute setting where it aids prognosis prediction and rehabilitation planning, Additional follow‐up imaging, such as angiography, should be dependent upon clinical findings.

Quantitative measures confirm the inverse relationship between lesion spiculation and detection of breast masses.

OBJECTIVE To identify specific mammographic appearances that reduce the mammographic detection of breast cancer. MATERIALS AND METHODS This study received institutional board review approval and all readers gave informed consent. A set of 60 mammograms each consisting of craniocaudal and mediolateral oblique projections were presented to 129 mammogram Breastscreen readers. The images consisted of 20 positive cases with single and multicentric masses in 16 and 4 cases, respectively (resulting in a total of 24 cancers), and readers were asked to identify and locate the lesions. Each lesion was then ranked according to a detectability rating (ie, the number of observers who correctly located the lesion divided by the total number of observers), and this was correlated with breast density, lesion size, and various descriptors of lesion shape and texture. RESULTS Negative and positive correlations between lesion detection and density (r = -0.64, P = .007) and size (r = 0.65, P = .005), respectively, were demonstrated. In terms of lesion size and shape, there were significant correlations between the probability of detection and area (r = 0.43, P = .04), perimeter (r = 0.66, P = .0004), lesion elongation (r = 0.49, P = .02), and lesion nonspiculation (r = 0.78, P < .0001). CONCLUSIONS The results of this study have identified specific lesion characteristics associated with shape that may contribute to reduced cancer detection. Mammographic sensitivity may be adversely affected without appropriate attention to spiculation.

The impact of mammographic density and lesion location on detection

The aim of this study is to examine the impact of breast density and lesion location on detection. A set of 55 mammographic images (23 abnormal images with 26 lesions and 32 normal images) were examined by 22 expert radiologists. The images were classified by an expert radiologist according to the Synoptic Breast Imaging Report of the National Breast Cancer Centre (NBCC) as having low mammographic density (D1<25% glandular and D2> 25-50% glandular) or high density (D3 51-75% glandular and D4> 75-glandular). The observers freely examined the images and located any malignancy using a 5-point confidence. Performance was defined using the following metrics: sensitivity, location sensitivity, specificity, receiver operating characteristic (ROC Az) curves and jackknife free-response receiver operator characteristics (JAFROC) figures of merit. Significant increases in sensitivity (p= 0.0174) and ROC (p=0.0001) values were noted for the higher density compared with lower density images according to NBCC classification. No differences were seen in radiologists’ performance between lesions within or outside the fibroglandular region. In conclusion, analysis of our data suggests that radiologists scored higher using traditional metrics in higher mammographic density images without any improvement in lesion localisation. Lesion location whether within or outside the fibroglandular region appeared to have no impact on detection abilities suggesting that if a masking effect is present the impact is minimal. Eye-tracking analyses are ongoing.

Does Expectation of Abnormality Affect the Search Pattern of Radiologists When Looking for Pulmonary Nodules

This experiment investigated whether there might be an effect on the visual search strategy of radiologists during image interpretation of the same adult chest radiographs when given different clinical information. Each of 17 experienced radiologists was asked to interpret a set of 57 (10 abnormal) posteroanterior chest images to identify the presence of pulmonary lesions using differing clinical information (leading to unknown, low and high expectations of prevalence). Eye position metrics (search time, dwell time and time to first fixation) were compared for normal and abnormal images, as well as between conditions. For all images, there was a significantly longer search time at high prevalence expectation compared to low prevalence expectation (W = 75.19, P = <0.0001). Mann–Whitney analysis of the abnormal images demonstrated that the dwell time on correctly identified lesions was significantly shorter at low prevalence expectation compared to both unknown (U = 364.5, P = 0.02) and high prevalence expectation (U = 397.0, P = 0.0002). Visual search patterns of radiologists appear to be affected by changing a priori information where such information fosters an expectation of abnormality.