John Papaioannou

Analysis of methods for reducing false positives in the automated detection of clustered microcalcifications in mammograms

Clustered microcalcifications are often the first sign of breast cancer in a mammogram. Nevertheless, all clustered microcalcifications are not found by an individual radiologist reading a mammogram. The use of a second reader may find those clusters of microcalcifications not found by the first reader, thereby improving the sensitivity of detecting clustered microcalcifications. Our laboratory has developed a computerized scheme for the detection of clustered microcalcifications, which can act like a second reader, that is undergoing clinical evaluation. This paper concerns the feature analysis stage of the computer scheme, which is designed to remove some of the false-computer detections. We have examined three methods of feature analysis, namely, rule based (the method currently used), an artificial neural network (ANN), and a combined method. In an independent database of 50 images, at a sensitivity of 83%, the average number of false positive (FP) detections per image was: 1.9 for rule-based, 1.6 for ANN, and 0.8 for the combined method. We demonstrate that the combined method performs best because each of the two stages eliminates different types of false positives.

Automated detection of microcalcification clusters for digital breast tomosynthesis using projection data only: A preliminary study

Digital breast tomosynthesis (DBT) is a promising modality for breast imaging in which an anisotropic volume image of the breast is obtained. We present an algorithm for computerized detection of microcalcification clusters (MCCs) for DBT. This algorithm operates on the projection views only. Therefore it does not depend on reconstruction, and is computationally efficient. The algorithm was developed using a database of 30 image sets with microcalcifications, and a control group of 30 image sets without visible findings. The patient data were acquired on the first DBT prototype at Massachusetts General Hospital. Algorithm sensitivity was estimated to be 0.86 at 1.3 false positive clusters, which is below that of current MCC detection algorithms for full-field digital mammography. Because of the small number of patient cases, algorithm parameters were not optimized and one linear classifier was used. An actual limitation of our approach may be that the signal-to-noise ratio in the projection images is too low for microcalcification detection. Furthermore, the database consisted of predominantly small MCC. This may be related to the image quality obtained with this first prototype.

Image feature analysis and computer-aided diagnosis in mammography: reduction of false-positive clustered microcalcifications using local edge-gradient analysis.

To improve the performance of a computerized scheme for detection of clustered microcalcifications in digitized mammograms, causes of detected false-positive microcalcification signals were analyzed. The false positives were grouped into four categories, namely, microcalcification like noise patterns, artifacts, linear patterns, and others. In an edge-gradient analysis, local edge-gradient values at signal-perimeter pixels of detected microcalcification signals were determined to eliminate false positives that look like subtle microcalcifications or are due to artifacts. In a linear-pattern analysis, the degree of linearity for linear patterns was determined from local gradient values from a set of linear templates oriented in 16 different directions. Threshold values for the edge-gradient analysis and the linear-pattern analysis were determined using a training database of 39 mammograms. It was possible to eliminate 59% and 25%, respectively, of 91 detected false-positive clusters with loss of only 3% of true-positive clusters. The combination of the two methods further improved the scheme in eliminating a total of 73% of the false-positive clusters with loss of 3% of true-positive clusters. Using these thresholds, the two methods were evaluated on another database of 50 mammograms. 62%, 31%, and 80% of the false-positive clusters were eliminated with loss of 3% of true-positive clusters or less, in the edge-gradient analysis, the linear-pattern analysis, and the combination of the two methods, respectively. The edge-gradient analysis and the linear-pattern analysis can reduce the false-positive detection rate, while maintaining a high level of the sensitivity.

Radial gradient-based segmentation of mammographic microcalcifications: Observer evaluation and effect on CAD performance

Precise segmentation of microcalcifications is essential in the development of accurate mammographic computer-aided diagnosis (CAD) schemes. We have designed a radial gradient-based segmentation method for microcalcifications, and compared it to both the region-growing segmentation method currently used in our CAD scheme and to the watershed segmentation method. Two observer studies were conducted to subjectively evaluate the proposed segmentation method. The first study (A) required observers to rate the segmentation accuracy on a 100-point scale. The second observer evaluation (B) was a preference study in which observers selected their preferred method from three displayed segmentation methods. In study A, the observers gave an average accuracy rating of 88 for the radial gradient-based and 50 for the region-growing segmentation method. In study B, the two observers selected the proposed method 56% and 62% of the time. We also investigated the effect of the proposed segmentation method on the performance of computerized classification scheme in differentiating malignant from benign clustered microcalcifications. The performances of the classification scheme using a linear discriminant analysis (LDA) or a Bayesian artificial neural network classifier both showed statistically significant improvements when using the proposed segmentation method. The areas under the receiver-operating characteristic curves for case-based performance when using the LDA classifier were 0.86 with the proposed segmentation method, 0.80 with the region-growing method, and 0.83 with the watershed method.

Clinically Missed Cancer: How Effectively Can Radiologists Use Computer-Aided Detection?

OBJECTIVE The purpose of this study was to determine the effectiveness with which radiologists can use computer-aided detection (CADe) to detect cancer missed at screening. MATERIALS AND METHODS An observer study was performed to measure the ability of radiologists to detect breast cancer on mammograms with and without CADe. The images in the study were from 300 analog mammographic examinations. In 234 cases the mammograms were read clinically as normal and free of cancer for at least 2 subsequent years. In the other 66 cases, cancers were missed clinically. In 256 cases, current and previous mammograms were available. Eight radiologists read the dataset and recorded a BI-RADS assessment, the location of the lesion, and their level of confidence that the patient should be recalled for diagnostic workup for each suspicious lesion. Jackknife alternative free-response receiver operating characteristic analysis was used. RESULTS The jackknife alternative free-response receiver operating characteristic figure of merit was 0.641 without aid and 0.659 with aid (p = 0.06; 95% CI, -0.001 to 0.036). The sensitivity increased 9.9% (95% CI, 3.4-19%) and the callback rate 12.1% (95% CI, 7.3-20%) with CADe. Both increases were statistically significant (p < 0.001). Radiologists on average ignored 71% of correct computer prompts. CONCLUSION Use of CADe can increase radiologist sensitivity 10% with a comparable increase in recall rate. There is potential for CADe to have a bigger clinical impact because radiologists failed to recognize a correct computer prompt in 71% of missed cancer cases [corrected].

Observers' ability to judge the similarity of clustered calcifications on mammograms

We are comparing two different methods for obtaining the radiologists’ subjective impression of similarity, for application in distinguishing benign from malignant lesions. Thirty pairs of mammographic clustered calcifications were used in this study. These 30 pairs were rated on a 5-point scale as to their similarity, where 1 was nearly identical and 5 was not at all similar. After this, all possible combinations of pairs of pairs were shown to the reader (n=435) and the reader selected which pair was most similar. This experiment was repeated by the observers with at least a week between reading sessions. Using analysis of variance, intra-class correlation coefficients (ICC) were calculated for both absolute scoring method and paired comparison method. In addition, for the paired comparison method, the coefficient of consistency within each reader was calculated. The average coefficient of consistence for the 4 readers was 0.88 (range 0.49-0.97). These results were statistically significant different from guessing at p << 0.0001. The ICC for intra-reader agreement was 0.51 (0.37-0.66 95% CI) for the absolute method and 0.82 (0.73-0.91 95% CI) for the paired comparison method. This difference was statistically significant (p=0.001). For the inter-reader agreement, the ICC for the absolute method was 0.39 (0.21-0.57 95% CI) and 0.37 (0.18-0.56 95% CI) for the paired comparison method. We conclude that humans are able to judge similarity of clustered calcifications in a meaningful way. Further, radiologists had greater intra-reader agreement when using the paired comparison method than when using an absolute rating scale. Differences in the criteria used by different observers to judge similarity and differences in interpreting which calcifications comprise the cluster can lead to low ICC values for inter-reader agreement for both methods.

Requirement of microcalcification detection for computerized classification of malignant and benign clustered microcalcifications

We are developing computerized schemes to detect clustered microcalcifications in mammograms, and to classify malignant versus benign microcalcifications. The purpose of this study is to investigate the effects on the performance of computer classification when results of computer-detected true microcalcifications and computer detected false-positive signals are used as input to the computer classification scheme. We found that when trained using manually identified microcalcifications, the computer classification performance was not degraded significantly when up to 60% of true microcalcifications were missed, and when false-positive signals made up approximately one half of the computer detection.

Can radiologists recognize that a computer has identified cancers that they have overlooked

For computer-aided detection (CADe) to be effective, the computer must be able to identify cancers that a radiologist misses clinically and the radiologist must be able to recognize that a cancer was missed when he or she reviews the computer output. There are several papers indicating CADe can detected clinically missed cancers. The purpose of this study is to examine whether radiologists can use the CADe output effectively to detect more cancers. Three-hundred mammographic cases, which included current and previous exams, were collected: 66 cases containing a missed cancer that was recognized in retrospect and 234 were normal cases. These were analyzed by a commercial CADe system. An observer study with eight MQSA-qualified radiologists was conducted using a sequential reading method. That is, the radiologist viewed the mammograms and scored the case. Then they reviewed the CADe output and rescored the case. The computer had a sensitivity of 55% with an average of 0.59 false detections per image. For all cancers (n=69), the radiologists had a sensitivity of 58% with no aid and 64% with aid (p=0.002). In cases where the computer detected the cancer in all views that the cancer was visible (n=17), the radiologists had a sensitivity of 74% unaided and increased to 85% aided (p=0.02). In cases where the computer missed the cancer in one view (n=21), the radiologists had a sensitivity of 65% unaided and 72% aided (p<0.001). The radiologists, on average, ignored 20% of all correct computer prompts.

Prospective Testing of a Clinical Mammography Workstation for CAD: Analysis of the First 10,000 Cases

For over ten years, we have been developing automated computerized schemes to assist radiologists in detecting breast cancer from mammograms. These detection schemes have been implemented on an “intelligent” mammography workstation that has been used prospectively on screening mammograms for over three years. The purpose of this study was to analyze the performance of the workstation in comparison to radiologists’ clinical interpretations of the same screening mammograms.

Eliminating false-positive microcalcification clusters in a mammography CAD scheme using a Bayesian neural network

We have applied a Bayesian Neural network (BNN) to the task of distinguishing between true-positive (TP) and false- positive (FP) detected clusters in a computer-aided diagnosis (CAD) scheme for detecting clustered microcalcifications in mammograms. Because BNNs can approximate ideal observer decision functions given sufficient training data, this approach should have better performance than our previous FP cluster elimination methods. Eight cluster-based features were extracted from the TP and FP clusters detected by the scheme in a training dataset of 39 mammograms. This set of features was used to train a BNN with eight input nodes, five hidden nodes, and one output node. The trained BNN was tested on the TP and FP clusters and detected by our scheme in an independent testing set of 50 mammograms. The BNN output was analyzed using ROC and FROC analysis. The detection scheme with BNN for FP cluster elimination had substantially better cluster sensitivity at low FP rates (below 0.8 FP clusters per image) than the original detection scheme without the BNN. Our preliminary research shows that a BNN can improve the performance of our scheme for detecting clusters of microcalcifications.