Jurgen Jacobs

Evaluation of clinical image processing algorithms used in digital mammography

Screening is the only proven approach to reduce the mortality of breast cancer, but significant numbers of breast cancers remain undetected even when all quality assurance guidelines are implemented. With the increasing adoption of digital mammography systems, image processing may be a key factor in the imaging chain. Although to our knowledge statistically significant effects of manufacturer-recommended image processings have not been previously demonstrated, the subjective experience of our radiologists, that the apparent image quality can vary considerably between different algorithms, motivated this study. This article addresses the impact of five such algorithms on the detection of clusters of microcalcifications. A database of unprocessed (raw) images of 200 normal digital mammograms, acquired with the Siemens Novation DR, was collected retrospectively. Realistic simulated microcalcification clusters were inserted in half of the unprocessed images. All unprocessed images were subsequently processed with five manufacturer-recommended image processing algorithms (Agfa Musica 1, IMS Raffaello Mammo 1.2, Sectra Mamea AB Sigmoid, Siemens OPVIEW v2, and Siemens OPVIEW v1). Four breast imaging radiologists were asked to locate and score the clusters in each image on a five point rating scale. The free-response data were analyzed by the jackknife free-response receiver operating characteristic (JAFROC) method and, for comparison, also with the receiver operating characteristic (ROC) method. JAFROC analysis revealed highly significant differences between the image processings (F = 8.51, p < 0.0001), suggesting that image processing strongly impacts the detectability of clusters. Siemens OPVIEW2 and Siemens OPVIEW1 yielded the highest and lowest performances, respectively. ROC analysis of the data also revealed significant differences between the processing but at lower significance (F = 3.47, p = 0.0305) than JAFROC. Both statistical analysis methods revealed that the same six pairs of modalities were significantly different, but the JAFROC confidence intervals were about 32% smaller than ROC confidence intervals. This study shows that image processing has a significant impact on the detection of microcalcifications in digital mammograms. Objective measurements, such as described here, should be used by the manufacturers to select the optimal image processing algorithm.

An improved method for simulating microcalcifications in digital mammograms

The assessment of the performance of a digital mammography system requires an observer study with a relatively large number of cases with known truth which is often difficult to assemble. Several investigators have developed methods for generating hybrid abnormal images containing simulated microcalcifications. This article addresses some of the limitations of earlier methods. The new method is based on digital images of needle biopsy specimens. Since the specimens are imaged separately from the breast, the microcalcification attenuation profile scan is deduced without the effects of over and underlying tissues. The resulting templates are normalized for image acquisition specific parameters and reprocessed to simulate microcalcifications appropriate to other imaging systems, with different x-ray, detector and image processing parameters than the original acquisition system. This capability is not shared by previous simulation methods that have relied on extracting microcalcifications from breast images. The method was validated by five experienced mammographers who compared 59 pairs of simulated and real microcalcifications in a two-alternative forced choice task designed to test if they could distinguish the real from the simulated lesions. They also classified the shapes of the microcalcifications according to a standardized clinical lexicon. The observed probability of correct choice was 0.415, 95% confidence interval (0.284, 0.546), showing that the radiologists were unable to distinguish the lesions. The shape classification revealed substantial agreement with the truth (mean kappa = 0.70), showing that we were able to accurately simulate the lesion morphology. While currently limited to single microcalcifications, the method is extensible to more complex clusters of microcalcifications and to three-dimensional images. It can be used to objectively assess an imaging technology, especially with respect to its ability to adequately visualize the morphology of the lesions, which is a critical factor in the benign versus malignant classification of a lesion detected in screening mammography.

Results of a European dose survey for mammography

For the dose study, a semi-automated method of data collection is used in this study. The participating centres were asked to fill out a spreadsheet with all necessary data and return it. For direct digital (DR) systems, the relevant data available in the DICOM header were used. All data is automatically added to a database and processed. The data were used to calculate the mean glandular dose for every image and for different thicknesses of polymethyl methacrylate phantoms using available conversion factors. Second-degree polynomials were fitted to the patient dose data and a reference dose curve was constructed for a range of thicknesses instead of a dose reference level at a single point. The dose reference curve rises from 1.57 mGy for a thickness of 30 mm to 2.50 mGy for 55 mm and 3.83 mGy for 75 mm. The results show centres that exceed this curve lie only in the lower or higher range of thicknesses and would remain undetected using a dose reference value in a single point. This gives better information to radiographers on where there is room for improvement of the dose levels in their system.

Preliminary validation of a new variable pattern for daily quality assurance of medical image display devices

This paper reports on a comparative study between the well-established test patterns for daily quality assurance (QA) of monitors of the American Association of Medical Physicists, Task Group 18 (AAPMtg18) and the Deutsches Institut für Normung e.V (DIN), and a newly proposed variable test pattern. A characteristic of the test patterns currently used for the QA of monitors is their static nature: The same test pattern is always used. This enables a learning effect that may bias the results over time. To address this problem we have developed a variable pattern for the quality assurance of monitors (MoniQA) that allows an evaluation of contrast visibility, geometric distortion, resolution, global image quality including uniformity, and artifacts. The test pattern includes randomly generated elements intended to prevent the observer from learning the test. Examples are random characters that have to be discriminated from the background to evaluate the threshold luminance difference and variable positions of different features in the test pattern. The newly proposed test patterns were generated and visualized on different viewing stations with a software tool developed in JAVA. In this study, we validated these patterns against the well-known AAPMtg18 and DIN test patterns on 22 monitors. The results showed that the MoniQA test can indicate the same monitor problems as the other well-known patterns and is significantly quicker to evaluate than the AAPMtg18 test patterns. The MoniQA pattern is a promising alternative for daily quality control of medical viewing stations.

One Year of Experience with Remote Quality Assurance of Digital Mammography Systems in the Flemish Breast Cancer Screening Program

The European Guidelines on Digital Mammography (EUREF) prescribe that regularly the homogeneity of the used digital systems should be tested. In a decentralized screening environment with centralized quality control (QC) supervision this can become a time consuming work. Therefore we developed a novel method to simplify remote QC. Exposures of a homogeneous plate of PMMA are made daily under clinical conditions and are sent to our locally installed analysis software. Several parameters are calculated for the complete image, for 6 reference regions of interest (ROIs) and for series of small adjacent ROIs all over the image. These calculated parameters are summarized in maps that are treated as thumbnail images. Analysis results are sent to the reference site where they are supervised by a trained physicist and compared with the results of previous tests. Several artifacts could be traced with the thumbnail images. These include: dirt on phosphor cassettes, scanline artifacts, scratches on the IP and burned-in markers for CR units. For DR units, increasing ghost image factors, lag images, crystallization of detector material, defective pixel artifacts and several electrical artifacts were noticed. Our initial experience indicates that failures with digital mammography devices can be traced remotely via thumbnail images of the above parameters that are electronically sent to our reference center, instead of the full-size image.

Evaluation of the global effect of anatomical background on microcalcifications detection

Purpose: 1/ To validate a method for simulating microcalcifications in mammography 2/ To evaluate the effect of anatomical background on visibility of (simulated) microcalcifications Materials and methods: Microcalcifications were extracted from the raw data of specimen from a stereotactic vacuum needle biopsy. The sizes of the templates varied from 200 μm to 1350μm and the peak contrast from 1.3% to 24%. Experienced breast imaging radiologists were asked to blindly evaluate images containing real and simulated lesions. Analysis was done using ROC methodology. The simulated lesions have been used for the creation of composite image datasets: 408 microcalcifications were simulated into 161 ROIs of 59 digital mammograms, having different anatomical backgrounds. Nine radiologists were asked to detect and rate them under conditions of free-search. A modified receiver operating characteristic study (FROC) was applied to find correlations between detectability and anatomical background. Results: 1/ The calculated area under the ROC curve, Az, was 0.52± 0.04. Simulated microcalcifications could not be distinguished from real ones. 2/ In the anatomical background classified as Category 1 (fatty), the detection fraction is the lowest (0.48), while for type 2,3,4 there is a gradually decrease (from 0.61 to 0.54) as the glandularity increases. The number of false positives is the highest for the background Category 1 (24%), compared to the other three types (16%). A 80% detectability is found for microcalcifications with a diameter > 400μm and a peak contrast >10%. Anatomic noise seems to limit detectability of large low contrast lesions, having a diameter >700μm.

Technical evaluation of a digital breast tomosynthesis system

This paper presents results of a technical evaluation of a digital breast tomosynthesis (DBT) system Projection images were used to assess x-ray focal spot size and lag as a function of detector air kerma (K) during a tomographic scan Modulation transfer function (MTF), normalized noise power spectrum (NNPS) and detective quantum efficiency (DQE) were used to obtain a quantitative assessment of detector operation, also from the projection image data Maximum 1st image lag was 4% at 20 mAs/projection Both edge and wire methods were used to assess MTF; in the direction of tube travel, MTF was reduced by a factor of 0.33 compared to 2D FFDM (static tube) Detector response results showed that detector gain is increased by a factor of 3.6 for DBT operation compared to 2D FFDM mode DQE measured at 28 kV W/Rh and 2 mm Al added filtration and a typical detector K of 23.8 μGy per projection was 0.5 at 0.5 mm−1, indicating quantum limited performance of the detector for DBT acquisitions The availability of projection image data enables the evaluation of important aspects of DBT detector performance in the field.

Toward an international consensus strategy for periodic quality control of digital breast tomosynthesis systems

As a collaborative effort between scientists affiliated with the American Association of Physicists in Medicine (AAPM) and the European reference center for breast cancer screening and diagnosis (EUREF), the Working Group on Phantoms for Breast Imaging (WGPBI) aims to develop phantoms and evaluation techniques for 2D & 3D breast imaging modalities. In the first phase of this collaboration, this project aimed to develop a phantom and associated procedure for constancy testing of digital breast tomosynthesis (DBT) systems. The procedure involves daily and weekly components. The daily evaluation is performed on a simple, homogenous PMMA plate of 4 cm thickness. For the weekly part, a new phantom has been designed consisting of a 45 mm thick homogeneous slab of PMMA with a set of spherical and rectangular inserts at specific 3D positions, and a thin wire positioned at a small angle to the plane of the detector. Quality control parameters are extracted from both projection images (if available) and reconstructed planes. The homogeneous phantom for daily QC allows a trend analysis of homogeneity and the assessment of detector artifacts. With the proposed phantom concept for weekly QC, the stability of the following parameters can be evaluated: the propagation and correlation of the noise in plane and across the reconstructed tomographic planes, lag, signal difference to noise ratio (SDNR) and signal to noise ratio (SNR), the geometry and the motion, effective thickness of the reconstructed planes, homogeneity, distance accuracy, frequency dependent SNR, and artifacts. Analysis of the DICOM header provides information on the stability of the automatic exposure control (AEC), exposure settings, and several system parameters. In an on-going study, the proposed strategy is being applied to five tomosynthesis systems both in Europe and in the US. In this paper we report on the specifics of the phantom, the QC procedure, the practicalities of remote data analysis, and the results of the initial trial.

Quality Control in Digital Mammography

An effective quality control system for digital mammography needs to evaluate the status of each stage of image formation — acquisition, processing and display. Such quality control benefits greatly from the ability to make more precise and reproducible measurements than was possible with film-screen systems. On the other hand, the greater variety of system designs and general lack of experience with different digital systems has complicated the introduction of quality control (QC) procedures. Those with extensive experience of QC in digital mammography have stressed the importance of checking regularly for artifacts in images of uniform test blocks for the early detection of any problems arising in the image acquisition stage, e.g. the detector. Although the tests for the subsequent stages of image processing and display are less well developed, they are of considerable importance and will be the focus of further work. Digital technology makes possible the automation of routine QC procedures and a method of doing this is described.

Development and implementation of a user friendly and automated environment for the creation of databases of digital mammograms with simulated microcalcifications

A database of raw composite mammograms containing simulated microcalcifications was generated. Databases can be used for technology assessment, quality assurance and comparison of different processing algorithms or different visualization modalities in digital mammography. Clinical mammograms were selected and fully documented for this scope. Microcalcifications were simulated in mammography images following a methodology developed and validated in an earlier work of our group. To create microcalcification templates, specimen containing lesions with different morphology types were acquired. From a basic set of (ideal) microcalcification templates, a set of specific templates for the systems under study was generated. The necessary input to do so is the system MTF and attenuation values of aluminum sheets with different thickness. In order to make the whole process less time consuming and applicable on a large scale, dedicated software tools for the creation of composite images have been developed. Automatic analysis of scores from observer performance study, in terms of microcalcification detectability on the composite images, is also implemented. We report on the functionalities foreseen in these new software tools. Simulated microcalcifications were successfully created and inserted in raw images of the Siemens Novation DR, the AGFA DM1000 and the AGFA CR MM2.0.