Henning Schubert

Content-based image retrieval in medical applications for picture archiving and communication systems

Picture archiving and communication systems (PACS) aim to efficiently provide the radiologists with all images in a suitable quality for diagnosis. Modern standards for digital imaging and communication in medicine (DICOM) comprise alphanumerical descriptions of study, patient, and technical parameters. Currently, this is the only information used to select relevant images within PACS. Since textual descriptions insufficiently describe the great variety of details in medical images, content-based image retrieval (CBIR) is expected to have a strong impact when integrated into PACS. However, existing CBIR approaches usually are limited to a distinct modality, organ, or diagnostic study. In this state-of-the-art report, we present first results implementing a general approach to content-based image retrieval in medical applications (IRMA) and discuss its integration into PACS environments. Usually, a PACS consists of a DICOM image server and several DICOM-compliant workstations, which are used by radiologists for reading the images and reporting the findings. Basic IRMA components are the relational database, the scheduler, and the web server, which all may be installed on the DICOM image server, and the IRMA daemons running on distributed machines, e.g., the radiologists’ workstations. These workstations can also host the web-based front-ends of IRMA applications. Integrating CBIR and PACS, a special focus is put on (a) location and access transparency for data, methods, and experiments, (b) replication transparency for methods in development, (c) concurrency transparency for job processing and feature extraction, (d) system transparency at method implementation time, and (e) job distribution transparency when issuing a query. Transparent integration will have a certain impact on diagnostic quality supporting both evidence-based medicine and case-based reasoning.

Extended Query Refinement for Medical Image Retrieval

The impact of image pattern recognition on accessing large databases of medical images has recently been explored, and content-based image retrieval (CBIR) in medical applications (IRMA) is researched. At the present, however, the impact of image retrieval on diagnosis is limited, and practical applications are scarce. One reason is the lack of suitable mechanisms for query refinement, in particular, the ability to (1) restore previous session states, (2) combine individual queries by Boolean operators, and (3) provide continuous-valued query refinement. This paper presents a powerful user interface for CBIR that provides all three mechanisms for extended query refinement. The various mechanisms of man–machine interaction during a retrieval session are grouped into four classes: (1) output modules, (2) parameter modules, (3) transaction modules, and (4) process modules, all of which are controlled by a detailed query logging. The query logging is linked to a relational database. Nested loops for interaction provide a maximum of flexibility within a minimum of complexity, as the entire data flow is still controlled within a single Web page. Our approach is implemented to support various modalities, orientations, and body regions using global features that model gray scale, texture, structure, and global shape characteristics. The resulting extended query refinement has a significant impact for medical CBIR applications.

Comparison of global features for categorization of medical images

We present an evaluation of methods for the automatic categorization of medical images. The properties of medical images render some otherwise very successful discriminate features for images (e.g. color) inapplicable. Therefore, we evaluate several feature types: texture, structure, and down-scaled representations. The classification is done using a nearest neighbor classifier with various distance measures as well as the automatic combination of classifier results. A corpus of 6,335 images selected arbitrarily from the clinical routine was encoded using a multi-axial, mono-hierarchical code. The reference categorization was done by experienced radiologists familiar with the code. The codes hierarchy allows the analysis of the automatic categorization performance (depending on the features and the classifier used) at different levels of differentiation. Experiments were done for 54 and 57 categories or 70 and 81 categories focussing on radiographs only or for all images, respectively. A maximum classification accuracy of 86% was obtained using the winner-takes-all rule and a one nearest neighbor classifier. Accuracy is increased to 93% and 95% if the correct category is only required to be within the 5 or 10 best matches, respectively. In this case, the best rate of 98% is obtained. This is sufficient for most applications in content-based image retrieval.

Quality of DICOM header information for image categorization

The widely used DICOM 3.0 imaging protocol specifies optional tags to store specific information on modality and body region within the header: Body Part Examined and Anatomic Structure. We investigate whether this information can be used for the automated categorization of medical images, as this is an important first step for medical image retrieval. Our survey examines the headers generated by four digital image modalities (2 CTs, 2 MRIs) in clinical routine at the Aachen University Hospital within a period of four months. The manufacturing dates of the modalities range from 1995 to 1999, with software revisions from 1999 and 2000. Only one modality sets the DICOM tag Body Part Examined. 90 out of 580 images (15.5%) contained false tag entries causing a wrong categorization. This result was verified during a second evaluation period of one month one year later (562 images, 15.3% error rate). The main reason is the dependency of the tag on the examination protocol of the modality, which controls all relevant parameters of the imaging process. In routine, the clinical personnel often applies an examination protocol outside its normal context to improve the imaging quality. This is, however, done without manually adjusting the categorization specific tag values. The values specified by DICOM for the tag Body Part Examined are insufficient to encode the anatomic region precisely. Thus, an automated categorization relying on DICOM tags alone is impossible.

Determining the View of Chest Radiographs

Automatic identification of frontal (posteroanterior/anteroposterior) vs. lateral chest radiographs is an important preprocessing step in computer-assisted diagnosis, content-based image retrieval, as well as picture archiving and communication systems. Here, a new approach is presented. After the radiographs are reduced substantially in size, several distance measures are applied for nearest-neighbor classification. Leaving-one-out experiments were performed based on 1,867 radiographs from clinical routine. For comparison to existing approaches, subsets of 430 and 5 training images are also considered. The overall best correctness of 99.7% is obtained for feature images of 32 × 32 pixels, the tangent distance, and a 5-nearest-neighbor classification scheme. Applying the normalized cross correlation function, correctness yields still 99.6% and 99.3% for feature images of 32 × 32 and 8 × 8 pixel, respectively. Remaining errors are caused by image altering pathologies, metal artifacts, or other interferences with routine conditions. The proposed algorithm outperforms existing but sophisticated approaches and is easily implemented at the same time.

Similarity of Medical Images Computed from Global Feature Vectors for Content-Based Retrieval

Global features describe the image content by a small number of numerical values, which are usually combined into a vector of less than 1,024 components. Since color is not present in most medical images, grey-scale and texture features are analyzed in order to distinguish medical imagery from vari- ous modalities. The reference data is collected arbitrarily from radiological rou- tine. Therefore, all anatomical regions and biological systems are present and all images have been captured in various directions. The ground truth is estab- lished by manually reference coding with respect to a mono-hierarchical unam- biguous coding scheme. Based on 6,335 images, experiments are performed for 54 and 57 categories or 70 and 81 categories focusing on radiographs only or considering all images, respectively. A maximum classification accuracy of 86% was obtained using the winner-takes-all rule and a one nearest neighbor classifier. If the correct category is only required to be within the 5 or 10 best matches, we yield a best rate of 98% using normalized cross correlation of small image icons.

A monohierarchical multiaxial classification code for medical images in content-based retrieval

Large efforts have been made for general applications of content-based image retrieval (CBIR). Established CBIR-systems globally evaluate color, texture, and also shape for retrieval. In medical imaging, local image characteristics are fundamental for image interpretation, which is based on a large amount of a-priori knowledge. Therefore, CBIR is rather seldom applied to medical images. Successful approaches strongly focus on a certain imaging modality and restrict queries to a well-defined diagnostic background. With respect to a general image retrieval in medical applications (IRMA), the system needs to determine the kind of image dealing with at a very early stage of processing to enable knowledge modeling required in further processing steps. In particular, 1. the imaging modality including technical parameters, 2. the orientation of the image with respect to the body, 3. the body region examined, and 4. the biological system under evaluation must be determined in order to select appropriate local techniques for image analysis. These four aspects build the axes of a general classification code for medical images. All axes are monohierarchically structured into three or five levels. The code is applied within the IRMA-project for medical image retrieval but also applicable for a great variety of applications in medical imaging in general.

Automatic detection of the view position of chest radiographs

Automatic identification of frontal (posteroanterior/anteroposterior) vs. lateral chest radiographs is an important preprocessing step in medical imaging. A recent approach by Amura et al. (Procs SPIE 2002; 4684: 308-315) is based on manual selection and combination of about 500 radiographs to generate as much as 24 templates by pixel-wise summing up the references, and a correctness rate of 99,99 % is reported. In order to design a fully automated procedure, 1,867 images were arbitrarily selected from clinical routine as reference for this work: 1,266 in frontal and 601 in lateral view position. The size of the radiographs varies between 2,000 and 4,000 pixels in each direction. Automatic categorization is done in two steps. At first, the image is reduced substantially in size. Regardless of the initial aspect ratio, a squared version is obtained, where the number h of pixels in both directions is a power of two. In the second step, the normalized cross correlation function at the optimal displacement is used for 5-nearest-neighbor classification. Leaving-one-out experiments were performed for h = 4, 8, 16, 32, and 64 resulting in mean correctness of 92.0 %, 99.3 %, 99.3 %, 99.6 % and 99.4 %, respectively. With respect to the approach of Amura et al., these results show that the determination of the view position of chest radiographs can be fully automated and substantially simplified if the correlation function is used directly for 5-NN classification.

Detailed image classification code for image retrieval of medical images (IRMA)

To support the automated classification of medical images an easy to use, tree-based, detailed examination code was developed. It consists of three parts: 1. technical code, 2. anatomical code, and 3. orientational code. The code was applied to about 6000 radiological images from daily routine, creating a well indexed database for testing of classifications. The code has been shown to be sufficient for the description of the images concerning image content.

Projektionsansichten zur Vereinfachung der Diagnose von multiplen Lungenrundherden in CT-Thorax-Aufnahmen

Neue Visualisierungstechniken zur Befundung von Rundherden in hochaufgelosten Thorax-CT-Daten aus MehrzeilenScannern werden vorgestellt. Sie basieren auf einer Segmentierung der Lungenflugel und verschiedenen Projektionsansichten sowie nichtlinearer, anatomische Reformatierung der Daten. Insbesondere die Detektion kleiner Rundherde in der Nahe der Pleura oder des Mediastinums kann damit erleichtert werden.