Ann Poulos

Effectiveness of feedback: the students’ perspective

While effective feedback has frequently been identified as a key strategy in learning and teaching, little known research has focused on students’ perceptions of feedback and the contribution feedback makes to students’ learning and teaching. This reported qualitative study aims to enrich our understanding of these perceptions and importantly to provide insight into the meaning of ‘effective’ when related to feedback. The study involved four focus groups of undergraduate students of varying levels and from a range of Schools completing degrees in the Faculty of Health Sciences, University of Sydney. Students’ perceptions relating to a definition of feedback, how they use it and preferences for delivery were prompted by the facilitators. Thematic analysis resulted in three key dimensions: perceptions of feedback, impact of feedback and credibility of feedback. The analysis demonstrated that effectiveness of feedback extends beyond mode of delivery and timeliness to include the credibility of the lecturer giving the feedback. The role of effective feedback includes not only enhancing learning and teaching but also facilitating the transition between school and university.

A review of methods of clinical image quality evaluation in mammography

PURPOSE Consistency in evaluation of mammography images in research and clinical practice is dependent on a standardised clinical image quality evaluation system. Currently two such systems are available-one developed by the American College of Radiology (ACR) and the other by the European Commission (EU guidelines). The purpose of this study was to review mammography clinical evaluation methods in research studies and their adherence to these systems. METHOD A total of 23 research articles were reviewed from the period 2000-2006, 11 of these studies used digital images. The focus of the review was the criteria and rating scales used. RESULTS Only 5 studies used either the ACR (3) or EU guidelines (2). The remainder included aspects of these systems together with a range of other criteria and rating scales. Variation was found in the categories of criteria used, number of criteria, the descriptors of the criteria and the instructions used to evaluate the criteria. Instructions were frequently not specific and open to individual interpretation. Although breast density is an important criterion of image quality and contributes to perception of breast lesions, inclusion of this criterion was not universal, and even when used the area of breast density to be evaluated was not identified, thus enhancing inter-observer variability. Scales that were absolute or relative were used for evaluation, all of which incorporated inconsistent numbers of steps. CONCLUSION Low adherence to ACR and EU Guidelines has resulted in considerable variation in the evaluation methods used in research studies. The implications of this variability are considerable both for evaluation of image quality in research outcomes and clinical practice.

Assessing reader performance in radiology, an imperfect science: lessons from breast screening.

The purpose of this article is to review the limitations associated with current methods of assessing reader accuracy in mammography screening programmes. Clinical audit is commonly used as a quality-assurance tool to monitor the performance of screen readers; however, a number of the metrics employed, such as recall rate as a surrogate for specificity, do not always accurately measure the intended clinical feature. Alternatively, standardized screening test sets, which benefit from ease of application, immediacy of results, and quicker assessment of quality improvement plans, suffer from experimental confounders, thus questioning the relevance of these laboratory-type screening test sets to clinical performance. Four key factors that impact on the external validity of screening test sets were identified: the nature and extent of scrutiny of ones action, the artificiality of the environment, the over-simplification of responses, and prevalence of abnormality. The impact of these factors on radiological and other contexts is discussed, and although it is important to acknowledge the benefit of standardized screening test sets, issues relating to the relevance of test sets to clinical activities remain. The degree of correlation between performance based on real-life clinical audit and performances at screen read test sets must be better understood and specific causal agents for any lack of correlation identified.

Reader practice in mammography screen reporting in Australia.

Reader variability is a problem in mammography image reporting and compromises the efficacy of screening programmes. The purpose of this exploratory study was to survey reader practice in reporting screening mammograms in Australia to identify aspects of practice that warrant further investigation. Mammography reporting practice and influences on concentration and attention were investigated by using an original questionnaire distributed to screen readers in Australia. A response rate of 71% (83 out of 117) was achieved. Demographic data indicated that the majority of readers were over 46 years of age (73%), have been reporting on screening mammograms for over 10 years (61%), take less than 1 min to report upon a screening mammogram examination (66%), report up to 200 examinations in a single session (83%) and take up to 2 h to report one session (61%). A majority report on more than 5000 examinations annually (66%); 93% of participants regard their search strategy as systematic, 87% agreed that their concentration can vary throughout a session, 64% agreed that the relatively low number of positives can lead to lapses in concentration and attention and almost all (94%) participants agreed that methods to maximise concentration should be explored. Participants identified a range of influences on concentration within their working environment including volume of images reported in one session, image types and aspects of the physical environment. This study has provided important evidence of the need to investigate adverse influences on concentration during mammography screen reporting

Mammography image quality: Model for predicting compliance with posterior nipple line criterion

PURPOSE To develop a model using measurements of pectoral muscle width and length together with the acceptability of the posterior nipple line criteria (PNL) to predict the acceptability of the presentation of the pectoral muscle in the mediolateral oblique view of the breast. METHOD A total of 400 mediolateral oblique mammogram images were randomly selected from BreastScreen NSW South West, Australia. Measurements of length and width of the pectoral muscle and the acceptability of the pectoral muscle position relative to the PNL were recorded. Data analysis involved logistic regression and ROC analysis to test the predictors of width and length and the performance of the model. The model was then used to predict the outcome of acceptable or unacceptable PNL criterion for each case. RESULTS The estimated odds ratio for an increase of 10mm was 1.98 (CI=1.68, 2.34) for the length predictor and 2.14 (CI=1.56, 2.93) for the width predictor. A cut off point of 0.6083 was derived from the training set and applied with the developed model to the test set. The area under the ROC curve was 0.9339 demonstrating an accurate model. CONCLUSION This paper describes a model to predict the acceptability of the PNL criterion using the width and length of the pectoral muscle. This model could be used in the automated assessment of image quality which has the potential to enhance the consistency in mammographic image quality evaluation. Optimising image quality contributes to increased accuracy in radiological interpretation, which maximises the early detection of breast cancer and potentially reduces mortality rates.

Evaluation of radiographers’ mammography screen‐reading accuracy in Australia

This study aimed to evaluate the accuracy of radiographers’ screen‐reading mammograms. Currently, radiologist workforce shortages may be compromising the BreastScreen Australia screening program goal to detect early breast cancer. The solution to a similar problem in the United Kingdom has successfully encouraged radiographers to take on the role as one of two screen‐readers. Prior to consideration of this strategy in Australia, educational and experiential differences between radiographers in the United Kingdom and Australia emphasise the need for an investigation of Australian radiographers’ screen‐reading accuracy.

The impact of mammographic density and lesion location on detection

The aim of this study is to examine the impact of breast density and lesion location on detection. A set of 55 mammographic images (23 abnormal images with 26 lesions and 32 normal images) were examined by 22 expert radiologists. The images were classified by an expert radiologist according to the Synoptic Breast Imaging Report of the National Breast Cancer Centre (NBCC) as having low mammographic density (D1<25% glandular and D2> 25-50% glandular) or high density (D3 51-75% glandular and D4> 75-glandular). The observers freely examined the images and located any malignancy using a 5-point confidence. Performance was defined using the following metrics: sensitivity, location sensitivity, specificity, receiver operating characteristic (ROC Az) curves and jackknife free-response receiver operator characteristics (JAFROC) figures of merit. Significant increases in sensitivity (p= 0.0174) and ROC (p=0.0001) values were noted for the higher density compared with lower density images according to NBCC classification. No differences were seen in radiologists’ performance between lesions within or outside the fibroglandular region. In conclusion, analysis of our data suggests that radiologists scored higher using traditional metrics in higher mammographic density images without any improvement in lesion localisation. Lesion location whether within or outside the fibroglandular region appeared to have no impact on detection abilities suggesting that if a masking effect is present the impact is minimal. Eye-tracking analyses are ongoing.

A comparison between the electronic magnification (EM) and true magnification (TM) of breast phantom images using a CDMAM phantom

PURPOSE To provide a comparison between the image quality of electronically magnified (EM) and geometric, or true, magnification (TM) mammographic images. MATERIALS AND METHODS One Computed Radiography (CR), one Digital Radiography (DR) and two screen-film (S-F) imaging systems were investigated. A Contrast-Detail Mammography (CDMAM) phantom was used as a test object. Three contact images and three sets of TM images with a magnification factor of 1.8 were taken on all systems. Software was used to zoom the contact images by a factor of 1.8 to produce EM images. Two observers evaluated all of the images. An Image Quality Figure and contrast detail curve were used to analyze the observer data and Mann-Whitney U-tests were performed to determine the statistical significance of the results. RESULTS No significant differences were found between soft copy and hard copy for any imaging modality. No significant difference in contrast detail detectability (CDD) was seen between EM images from the two digital systems and TM images on S-F systems. The results for the DR EM images and S-F TM images also showed no differences. The CDD of DR TM images was significantly better than both EM and S-F TM images. CONCLUSION Digitally zoomed images offer the same level of CDD as S-F TM images, and so may be viably used in their place. DR systems offer greater CDD than conventional S-F images, when comparing the TM images. This implies that doses can be greatly reduced for TM views using DR systems, while maintaining acceptable image quality.

Mammography image quality and evidence based practice: Analysis of the demonstration of the inframammary angle in the digital setting

AIM The aim of this study is to determine the clinical rates of the demonstration of the inframammary angle (IMA) on the mediolateral oblique (MLO) view of the breast on digital mammograms and to compare the outcomes with current accreditation standards for compliance. Relationships between the IMA, age, the posterior nipple line (PNL) and compressed breast thickness will be identified and the study outcomes validated using appropriate analyses of inter-reader and inter-rater reliability and variability. Differences in left versus right data were also investigated. METHOD A quantitative retrospective study of 2270 randomly selected paired digital mammograms performed by BreastScreen NSW was undertaken. Data was collected by direct measurement and visual analysis. Intra-class correlation analyses were used to evaluate inter- and intra-rater reliability. RESULTS The IMA was demonstrated on 52.4% of individual and 42.6% of paired mammograms. A linear relationship was found between the posterior nipple line (PNL) and age (p-value <0.001). The PNL was predicted to increase by 0.48 mm for every one year increment in age. The odds of demonstrating the IMA reduced by 2% for every one year increase in age (p-value = 0.001); are 0.4% higher for every 1 mm increase in PNL (p-value = 0.001) and 1.6% lower for every 1 mm increase in compressed breast thickness, (p-value<0.001). There was high inter- and intra-rater reliability for the PNL while there was 100% agreement for the demonstration of the IMA. CONCLUSION Analysis of the demonstration of the IMA indicates clinically achievable rates (42.6%) well below that required for compliance (50%-75%) to known worldwide accreditation standards for screening mammography. These standards should be aligned to the reported evidence base. Visualisation of the IMA is impacted negatively by increasing age and compressed breast thickness but positively by breast size (PNL).

Evaluating radiographers' diagnostic accuracy in screen-reading mammograms: what constitutes a quality study?

The aim of this study was to first evaluate the quality of studies investigating the diagnostic accuracy of radiographers as mammogram screen‐readers and then to develop an adapted tool for determining the quality of screen‐reading studies.