Mark Oliver Güld

Content-based image retrieval in medical applications for picture archiving and communication systems

Picture archiving and communication systems (PACS) aim to efficiently provide the radiologists with all images in a suitable quality for diagnosis. Modern standards for digital imaging and communication in medicine (DICOM) comprise alphanumerical descriptions of study, patient, and technical parameters. Currently, this is the only information used to select relevant images within PACS. Since textual descriptions insufficiently describe the great variety of details in medical images, content-based image retrieval (CBIR) is expected to have a strong impact when integrated into PACS. However, existing CBIR approaches usually are limited to a distinct modality, organ, or diagnostic study. In this state-of-the-art report, we present first results implementing a general approach to content-based image retrieval in medical applications (IRMA) and discuss its integration into PACS environments. Usually, a PACS consists of a DICOM image server and several DICOM-compliant workstations, which are used by radiologists for reading the images and reporting the findings. Basic IRMA components are the relational database, the scheduler, and the web server, which all may be installed on the DICOM image server, and the IRMA daemons running on distributed machines, e.g., the radiologists’ workstations. These workstations can also host the web-based front-ends of IRMA applications. Integrating CBIR and PACS, a special focus is put on (a) location and access transparency for data, methods, and experiments, (b) replication transparency for methods in development, (c) concurrency transparency for job processing and feature extraction, (d) system transparency at method implementation time, and (e) job distribution transparency when issuing a query. Transparent integration will have a certain impact on diagnostic quality supporting both evidence-based medicine and case-based reasoning.

A generic concept for the implementation of medical image retrieval systems

This work presents mechanisms to support the development and installation of content-based image retrieval in medical applications (IRMA). A strict separation of feature extraction, feature storage, feature comparison, and the user interfaces is suggested. The concept and implementation of a system following these guidelines is described. The system allows to reuse implemented components in different retrieval algorithms, which improves software quality, shortens the development cycle for applications, and allows to establish standardized end-user interfaces.

Overview of the CLEF 2009 medical image annotation track

This paper describes the last round of the medical image annotation task in ImageCLEF 2009. After four years, we defined the task as a survey of all the past experience. Seven groups participated to the challenge submitting nineteen runs. They were asked to train their algorithms on 12677 images, labelled according to four different settings, and to classify 1733 images in the four annotation frameworks. The aim is to understand how each strategy answers to the increasing number of classes and to the unbalancing. A plain classification scheme using support vector machines and local descriptors outperformed the other methods.

Extended Query Refinement for Medical Image Retrieval

The impact of image pattern recognition on accessing large databases of medical images has recently been explored, and content-based image retrieval (CBIR) in medical applications (IRMA) is researched. At the present, however, the impact of image retrieval on diagnosis is limited, and practical applications are scarce. One reason is the lack of suitable mechanisms for query refinement, in particular, the ability to (1) restore previous session states, (2) combine individual queries by Boolean operators, and (3) provide continuous-valued query refinement. This paper presents a powerful user interface for CBIR that provides all three mechanisms for extended query refinement. The various mechanisms of man–machine interaction during a retrieval session are grouped into four classes: (1) output modules, (2) parameter modules, (3) transaction modules, and (4) process modules, all of which are controlled by a detailed query logging. The query logging is linked to a relational database. Nested loops for interaction provide a maximum of flexibility within a minimum of complexity, as the entire data flow is still controlled within a single Web page. Our approach is implemented to support various modalities, orientations, and body regions using global features that model gray scale, texture, structure, and global shape characteristics. The resulting extended query refinement has a significant impact for medical CBIR applications.

Statistical Image Object Recognition using Mixture Densities

In this paper, we present a mixture density based approach to invariant image object recognition. To allow for a reliable estimation of the mixture parameters, the dimensionality of the feature space is optionally reduced by applying a robust variant of linear discriminant analysis. Invariance to affine transformations is achieved by incorporating invariant distance measures such as tangent distance. We propose an approach to estimating covariance matrices with respect to image variabilities as well as a new approach to combined classification, called the virtual test sample method. Application of the proposed classifier to the well known US Postal Service handwritten digits recognition task (USPS) yields an excellent error rate of 2.2%. We also propose a simple, but effective approach to compensate for local image transformations, which significantly increases the performance of tangent distance on a database of 1,617 medical radiographs taken from clinical daily routine.

Content-based retrieval of medical images by combining global features

A combination of several classifiers using global features for the content description of medical images is proposed. Beside well known texture histogram features, downscaled representations of the original images are used, which preserve spatial information and utilize distance measures which are robust with regard to common variations in radiation dose, translation, and local deformation. These features were evaluated for the annotation task and the retrieval task in ImageCLEF 2005 without using additional textual information or query refinement mechanisms. For the annotation task, a categorization rate of 86.7% was obtained, which ranks second among all submissions. When applied in the retrieval task, the image content descriptors yielded a mean average precision (MAP) of 0.0751, which is rank 14 of 28 submitted runs. As the image deformation model is not fit for interactive retrieval tasks, two mechanisms are evaluated with regard to the trade-off between loss of accuracy and speed increase: hierarchical filtering and prototype selection.

Content-based queries on the casimage database within the IRMA framework

Recent research has suggested that there is no general similarity measure, which can be applied on arbitrary databases without any parameterization. Hence, the optimal combination of similarity measures and parameters must be identified for each new image repository. This optimization loop is time consuming and depends on the experience of the designer as well as the knowledge of the medical expert. It would be useful if results that have been obtained for one data set can be transferred to another without extensive re-design. This transfer is vital if content-based image retrieval is integrated into complex environments such as picture archiving and communication systems. The image retrieval in medical applications (IRMA) project defines a framework that strictly separates data administration and application logic. This permits an efficient transfer of the data abstraction of one database on another without re-designing the software. In the ImageCLEF competition, the query performance was evaluated on the CasImage data set without optimization of the feature combination successfully applied to the IRMA corpus. IRMA only makes use of basic features obtained from grey-value representations of the images without additional textual annotations. The results indicate that transfer of parameterization is possible without time consuming parameter adaption and significant loss of retrieval quality.

Content-based image retrieval by matching hierarchical attributed region adjacency graphs

Content-based image retrieval requires a formal description of visual information. In medical applications, all relevant biological objects have to be represented by this description. Although color as the primary feature has proven successful in publicly available retrieval systems of general purpose, this description is not applicable to most medical images. Additionally, it has been shown that global features characterizing the whole image do not lead to acceptable results in the medical context or that they are only suitable for specific applications. For a general purpose content-based comparison of medical images, local, i.e. regional features that are collected on multiple scales must be used. A hierarchical attributed region adjacency graph (HARAG) provides such a representation and transfers image comparison to graph matching. However, building a HARAG from an image requires a restriction in size to be computationally feasible while at the same time all visually plausible information must be preserved. For this purpose, mechanisms for the reduction of the graph size are presented. Even with a reduced graph, the problem of graph matching remains NP-complete. In this paper, the Similarity Flooding approach and Hopfield-style neural networks are adapted from the graph matching community to the needs of HARAG comparison. Based on synthetic image material build from simple geometric objects, all visually similar regions were matched accordingly showing the frameworks general applicability to content-based image retrieval of medical images.

Similarity of Medical Images Computed from Global Feature Vectors for Content-Based Retrieval

Global features describe the image content by a small number of numerical values, which are usually combined into a vector of less than 1,024 components. Since color is not present in most medical images, grey-scale and texture features are analyzed in order to distinguish medical imagery from vari- ous modalities. The reference data is collected arbitrarily from radiological rou- tine. Therefore, all anatomical regions and biological systems are present and all images have been captured in various directions. The ground truth is estab- lished by manually reference coding with respect to a mono-hierarchical unam- biguous coding scheme. Based on 6,335 images, experiments are performed for 54 and 57 categories or 70 and 81 categories focusing on radiographs only or considering all images, respectively. A maximum classification accuracy of 86% was obtained using the winner-takes-all rule and a one nearest neighbor classifier. If the correct category is only required to be within the 5 or 10 best matches, we yield a best rate of 98% using normalized cross correlation of small image icons.

Interfacing Global and Local CBIR Systems for Medical Image Retrieval

Contemporary picture archiving and communication systems are limited in managing large and varied image collections, because content-based image retrieval (CBIR) methods are unavailable. In this paper, an XML-based data and resource exchange framework is defined using open standards and software to enable specialized CBIR systems to act as geographically distributed toolkits. The approach enables communication and collaboration between two or more geographically separated complementary systems with possibly different architectures and developed on different platforms, and specialized for different image modalities and characteristics. The resulting synergy provides the user with a rich functionality operating within a familiar Web browser interface, making the combined system portable and independent of location and underlying user operating systems. We describe the coupling of the Image Retrieval in Medical Applications (IRMA) system and the Spine Pathology and Image Retrieval System (SPIRS) as proof of this concept.