Michael Kohnen

Content-based image retrieval in medical applications: a novel multistep approach

In the past few years, immense improvement was obtained in the field of content-based image retrieval. Nevertheless, existing systems still fail when applied to medical image databases. Simple feature-extraction algorithms that operate on the entire image for characterization of color, texture, or shape cannot be related to the descriptive semantics of medical knowledge that is extracted from images by human experts.

Content-based image retrieval in medical applications for picture archiving and communication systems

Picture archiving and communication systems (PACS) aim to efficiently provide the radiologists with all images in a suitable quality for diagnosis. Modern standards for digital imaging and communication in medicine (DICOM) comprise alphanumerical descriptions of study, patient, and technical parameters. Currently, this is the only information used to select relevant images within PACS. Since textual descriptions insufficiently describe the great variety of details in medical images, content-based image retrieval (CBIR) is expected to have a strong impact when integrated into PACS. However, existing CBIR approaches usually are limited to a distinct modality, organ, or diagnostic study. In this state-of-the-art report, we present first results implementing a general approach to content-based image retrieval in medical applications (IRMA) and discuss its integration into PACS environments. Usually, a PACS consists of a DICOM image server and several DICOM-compliant workstations, which are used by radiologists for reading the images and reporting the findings. Basic IRMA components are the relational database, the scheduler, and the web server, which all may be installed on the DICOM image server, and the IRMA daemons running on distributed machines, e.g., the radiologists’ workstations. These workstations can also host the web-based front-ends of IRMA applications. Integrating CBIR and PACS, a special focus is put on (a) location and access transparency for data, methods, and experiments, (b) replication transparency for methods in development, (c) concurrency transparency for job processing and feature extraction, (d) system transparency at method implementation time, and (e) job distribution transparency when issuing a query. Transparent integration will have a certain impact on diagnostic quality supporting both evidence-based medicine and case-based reasoning.

Quality of DICOM header information for image categorization

The widely used DICOM 3.0 imaging protocol specifies optional tags to store specific information on modality and body region within the header: Body Part Examined and Anatomic Structure. We investigate whether this information can be used for the automated categorization of medical images, as this is an important first step for medical image retrieval. Our survey examines the headers generated by four digital image modalities (2 CTs, 2 MRIs) in clinical routine at the Aachen University Hospital within a period of four months. The manufacturing dates of the modalities range from 1995 to 1999, with software revisions from 1999 and 2000. Only one modality sets the DICOM tag Body Part Examined. 90 out of 580 images (15.5%) contained false tag entries causing a wrong categorization. This result was verified during a second evaluation period of one month one year later (562 images, 15.3% error rate). The main reason is the dependency of the tag on the examination protocol of the modality, which controls all relevant parameters of the imaging process. In routine, the clinical personnel often applies an examination protocol outside its normal context to improve the imaging quality. This is, however, done without manually adjusting the categorization specific tag values. The values specified by DICOM for the tag Body Part Examined are insufficient to encode the anatomic region precisely. Thus, an automated categorization relying on DICOM tags alone is impossible.

Determining the View of Chest Radiographs

Automatic identification of frontal (posteroanterior/anteroposterior) vs. lateral chest radiographs is an important preprocessing step in computer-assisted diagnosis, content-based image retrieval, as well as picture archiving and communication systems. Here, a new approach is presented. After the radiographs are reduced substantially in size, several distance measures are applied for nearest-neighbor classification. Leaving-one-out experiments were performed based on 1,867 radiographs from clinical routine. For comparison to existing approaches, subsets of 430 and 5 training images are also considered. The overall best correctness of 99.7% is obtained for feature images of 32 × 32 pixels, the tangent distance, and a 5-nearest-neighbor classification scheme. Applying the normalized cross correlation function, correctness yields still 99.6% and 99.3% for feature images of 32 × 32 and 8 × 8 pixel, respectively. Remaining errors are caused by image altering pathologies, metal artifacts, or other interferences with routine conditions. The proposed algorithm outperforms existing but sophisticated approaches and is easily implemented at the same time.

Hierarchical feature clustering for content-based retrieval in medical image databases

In this paper we describe the construction of hierarchical feature clustering and show how to overcome general problems of region growing algorithms such as seed point selection and processing order. Access to medical knowledge inherent in medical image databases requires content-based descriptions to allow non-textual retrieval, e.g., for comparison, statistical inquiries, or education. Due to varying medical context and questions, data structures for image description must provide all visually perceivable regions and their topological relationships, which poses one of the major problems for content extraction. In medical applications main criteria for segmenting images are local features such as texture, shape, intensity extrema, or gray values. For this new approach, these features are computed pixel-based and neighboring pixels are merged if the Euclidean distance of corresponding feature vectors is below a threshold. Thus, the planar adjacency of clusters representing connected image partitions is preserved. A cluster hierarchy is obtained by iterating and recording the adjacency merging. The resulting inclusion and neighborhood relations of the regions form a hierarchical region adjacency graph. This graph represents a multiscale image decomposition and therefore an extensive content description. It is examined with respect to application in daily routine by testing invariance against transformation, run time behavior, and visual quality For retrieval purposes, a graph can be matched with graphs of other images, where the quality of the matching describes the similarity of the images.

Skeletal maturity determination from hand radiograph by model-based analysis

Derived from a model based segmentation algorithm for hand radiographs proposed in our former work we now present a method to determine skeletal maturity by an automated analysis of regions of interest (ROI). These ROIs including the epiphyseal and carpal bones, which are most important for skeletal maturity determination, can be extracted out of the radiograph by knowledge based algorithms.

A monohierarchical multiaxial classification code for medical images in content-based retrieval

Large efforts have been made for general applications of content-based image retrieval (CBIR). Established CBIR-systems globally evaluate color, texture, and also shape for retrieval. In medical imaging, local image characteristics are fundamental for image interpretation, which is based on a large amount of a-priori knowledge. Therefore, CBIR is rather seldom applied to medical images. Successful approaches strongly focus on a certain imaging modality and restrict queries to a well-defined diagnostic background. With respect to a general image retrieval in medical applications (IRMA), the system needs to determine the kind of image dealing with at a very early stage of processing to enable knowledge modeling required in further processing steps. In particular, 1. the imaging modality including technical parameters, 2. the orientation of the image with respect to the body, 3. the body region examined, and 4. the biological system under evaluation must be determined in order to select appropriate local techniques for image analysis. These four aspects build the axes of a general classification code for medical images. All axes are monohierarchically structured into three or five levels. The code is applied within the IRMA-project for medical image retrieval but also applicable for a great variety of applications in medical imaging in general.

Detailed image classification code for image retrieval of medical images (IRMA)

To support the automated classification of medical images an easy to use, tree-based, detailed examination code was developed. It consists of three parts: 1. technical code, 2. anatomical code, and 3. orientational code. The code was applied to about 6000 radiological images from daily routine, creating a well indexed database for testing of classifications. The code has been shown to be sufficient for the description of the images concerning image content.

Model based analysis of chest radiographs

Chest radiographs represent a difficult class of images concerning automatic analysis with image processing methods. In our former work we presented a model based method to detect the rib borders and implemented a compensation algorithm of the rib structures. Recently we developed an improved method for rib border detection and algorithms to find the objects like chest border, vertebral spine, heart and intravascular catheter within a model driven approach. The determined borders of these objects allow further analysis and image enhancement for diagnose assistance.

Knowledge-based automated feature extraction to categorize secondary digitized radiographs

An essential part of the IRMA-project (Image Retrieval in Medical Applications) is the categorization of digitized images into predefined classes using a combination of different independent features. To obtain an automated and content-based categorization, the following features are extracted from the image data: Fourier coefficients of normalized projections are computed to supply a scale- and translation-invariant description. Furthermore, histogram information and Co-occurrence matrices are calculated to supply information about the gray value distribution and textural information. But the key part of the feature extraction is the shape information of the objects represented by an Active Shape Model. The Active Shape Model supports various form variations given by a representative training set; we use one particular Active Shape Model for each image class. These different Active Shape Models are matched on preprocessed image data with a simulated annealing optimization. The different extracted features were chosen with regard to the different characteristics of the image content. They give a comprehensive description of image content using only few different features. Using this combination of different features for categorization results in a robust classification of image data, which is a basic step towards medical archives that allow retrieval results for queries of diagnostic relevance.