L. Rodney Long

Ontology of Gaps in Content-Based Image Retrieval

Content-based image retrieval (CBIR) is a promising technology to enrich the core functionality of picture archiving and communication systems (PACS). CBIR has a potential for making a strong impact in diagnostics, research, and education. Research as reported in the scientific literature, however, has not made significant inroads as medical CBIR applications incorporated into routine clinical medicine or medical research. The cause is often attributed (without supporting analysis) to the inability of these applications in overcoming the “semantic gap.” The semantic gap divides the high-level scene understanding and interpretation available with human cognitive capabilities from the low-level pixel analysis of computers, based on mathematical processing and artificial intelligence methods. In this paper, we suggest a more systematic and comprehensive view of the concept of “gaps” in medical CBIR research. In particular, we define an ontology of 14 gaps that addresses the image content and features, as well as system performance and usability. In addition to these gaps, we identify seven system characteristics that impact CBIR applicability and performance. The framework we have created can be used a posteriori to compare medical CBIR systems and approaches for specific biomedical image domains and goals and a priori during the design phase of a medical CBIR application, as the systematic analysis of gaps provides detailed insight in system comparison and helps to direct future research.

Histology image analysis for carcinoma detection and grading

This paper presents an overview of the image analysis techniques in the domain of histopathology, specifically, for the objective of automated carcinoma detection and classification. As in other biomedical imaging areas such as radiology, many computer assisted diagnosis (CAD) systems have been implemented to aid histopathologists and clinicians in cancer diagnosis and research, which have been attempted to significantly reduce the labor and subjectivity of traditional manual intervention with histology images. The task of automated histology image analysis is usually not simple due to the unique characteristics of histology imaging, including the variability in image preparation techniques, clinical interpretation protocols, and the complex structures and very large size of the images themselves. In this paper we discuss those characteristics, provide relevant background information about slide preparation and interpretation, and review the application of digital image processing techniques to the field of histology image analysis. In particular, emphasis is given to state-of-the-art image segmentation methods for feature extraction and disease classification. Four major carcinomas of cervix, prostate, breast, and lung are selected to illustrate the functions and capabilities of existing CAD systems.

Age-Related Changes of the Cervix Influence Human Papillomavirus Type Distribution

Approximately 15 human papillomavirus (HPV) types cause virtually all cervical cancer whereas other HPV types are unrelated to cancer. We were interested in whether some noncarcinogenic types differ from carcinogenic in their affinity for the cervical transformation zone, where nearly all HPV-induced cancers occur. To examine this possibility, we tested cervical specimens from 8,374 women without cervical precancer and cancer participating in a population-based study in Guanacaste for >40 HPV types using PCR. We compared age-group specific prevalences of HPV types of the alpha9 species, which are mainly carcinogenic and include HPV16, to the genetically distinct types of the alpha3/alpha15 species (e.g., HPV71), which are noncarcinogenic and common in vaginal specimens from hysterectomized women. We related HPV detection of each group to the location of the junction between the squamous epithelium of the ectocervix and vagina and the columnar epithelium of the endocervical canal. Models evaluated the independent effects of amount of exposed columnar epithelium (ectopy) and age on the presence of alpha9 or alpha3/alpha15 types. Prevalence of alpha9 types (7.6%) peaked in the youngest women, declined in middle-aged women, and then increased slightly in older women. By contrast, prevalence of alpha3/alpha15 types (7.6%) tended to remain invariant or to increase with increasing age. Detection of alpha9 infections increased (P(trend) < 0.0005) but alpha3/alpha15 infections decreased (P(trend) < 0.0005) with increasing exposure of the columnar epithelia. Older age and decreasing cervical ectopy were independently positively associated with having an alpha3/alpha15 infection compared with having an alpha9 infection. These patterns need to be confirmed in other studies and populations. We suggest that these genetically distinct groups of HPV types may differ in tissue preferences, which may contribute to their differences in carcinogenic potential.

SPIRS: A Web-based Image Retrieval System for Large Biomedical Databases

PURPOSE With the increasing use of images in disease research, education, and clinical medicine, the need for methods that effectively archive, query, and retrieve these images by their content is underscored. This paper describes the implementation of a Web-based retrieval system called SPIRS (Spine Pathology & Image Retrieval System), which permits exploration of a large biomedical database of digitized spine X-ray images and data from a national health survey using a combination of visual and textual queries. METHODS SPIRS is a generalizable framework that consists of four components: a client applet, a gateway, an indexing and retrieval system, and a database of images and associated text data. The prototype system is demonstrated using text and imaging data collected as part of the second U.S. National Health and Nutrition Examination Survey (NHANES II). Users search the image data by providing a sketch of the vertebral outline or selecting an example vertebral image and some relevant text parameters. Pertinent pathology on the image/sketch can be annotated and weighted to indicate importance. RESULTS During the course of development, we explored different algorithms to perform functions such as segmentation, indexing, and retrieval. Each algorithm was tested individually and then implemented as part of SPIRS. To evaluate the overall system, we first tested the systems ability to return similar vertebral shapes from the database given a query shape. Initial evaluations using visual queries only (no text) have shown that the system achieves up to 68% accuracy in finding images in the database that exhibit similar abnormality type and severity. Relevance feedback mechanisms have been shown to increase accuracy by an additional 22% after three iterations. While we primarily demonstrate this system in the context of retrieving vertebral shape, our framework has also been adapted to search a collection of 100,000 uterine cervix images to study the progression of cervical cancer. CONCLUSIONS SPIRS is automated, easily accessible, and integratable with other complementary information retrieval systems. The system supports the ability for users to intuitively query large amounts of imaging data by providing visual examples and text keywords and has beneficial implications in the areas of research, education, and patient care.

Similarity measurement using polygon curve representation and Fourier descriptors for shape-based vertebral image retrieval

Shape-based retrieval of vertebral x-ray images is a challenging task because of high similarity among the vertebral shapes. Most techniques, such as global shape properties or scale space filtering, lose or fail to detect local details. As the result of this shortfall, the number of retrieved images is so high that the retrieval result is sometimes meaningless. To retrieve a small number of best matched images, shape representation and similarity measurement techniques must distinguish shapes with minor variations. The main challenge of shape-based retrieval is to define a shape representation method that is invariant with respect to rotation, translation, scaling, and the curve starting point shift. In this research, a polygon curve evolution technique was developed for smoothing polygon curves and reducing the number of data points while preserving the significant pathology of the shape. The x and y coordinates of the simplified boundary points were then converted into a bend angle versus normalized curvature length function to represent the curve. Finally, the Fourier descriptors of the shape representation were calculated for similarity measurement. This approach meets the invariance requirements and has been proved to be efficient and accurate.

Digital tools for collecting data from cervigrams for research and training in colposcopy.

Abstract: Colposcopy is a critical part of gynecologic practice but has documented deficiencies, including lack of correlation between the colposcopic appearance and the severity of underlying neoplasia, limited reproducibility, and difficulty in the optimal placement of colposcopically directed biopsies. In a collaborative effort to improve colposcopy, we are analyzing digitized cervigram images from National Cancer Institute-funded studies. Specifically, the National Cancer Institute has collected close to 100,000 cervigrams, digitized to create a database of images of the uterine cervix for research, training, and education. In addition to the cervigram images, this database contains clinical, cytologic, and molecular information at multiple examinations of 15,000 women, with password and ID labeling strategies to protect patient privacy. The National Library of Medicine has designed two web-accessible software tools. The Boundary Marking Tool allows experts on colposcopy to perform an evaluation of the pictures and to mark boundary regions of normal and abnormal regions of the uterine cervix; these evaluations are collected and saved in the database. The Multimedia Database Tool enables retrieval of test and image biomedical data according to specific queries, for example, all women with cervical intraepithelial neoplasia 3 whose cytologic results are atypical squamous cells of undetermined significance. The resource soon will be available as an open resource, via a teaching tool coordinated by a database manager, which will permit a variety of applications for teaching and research. In this article, we describe the perceived need for the resource and its components.

WebMIRS: Web-based medical information retrieval system

At the Lister Hill National Center for Biomedical Communications, a research and development division of the National Library of Medicine (NLM), we are developing a prototype multimedia database system to provide World Wide Web access to biomedical databases. WebMIRS (Web-based Medical Information Retrieval System) will allow access to databases containing text and images and will allow database query by standard SQL, by image content, or by a combination of the two. The system is being developed in the form of Java applets, which will communicate with the Informix DBMS on an NLM Sun workstation running the Solaris operating system. The system architecture will allow access from any hardware platform, which supports a Java-enabled Web browser, such as Netscape or Internet Explorer. Initial databases will include data from two national health surveys conducted by the National Center for Health Statistics (NCHS), and will include x-ray images from those surveys. In addition to describing in- house research in database access systems, this paper describes ongoing work toward querying by image content. Image content search capability will include capability to search for x-ray images similar to an input image with respect to vertebral morphometry used to characterize features such as fractures and disc space narrowing.

Biomedical information from a national collection of spine x-rays: film to content-based retrieval

We summarize research and development for the extraction and distribution of biomedical information from a collection of 17,000 spine x-ray images collected by the second National Health and Nutrition Examination Survey (NHANES II). We present a history of the technical milestones of this work, including the data collection as film, digitization, quality control, archiving technology, database organization, medical expert content evaluation, and Web data distribution. We conclude by presenting our current work in content-based image retrieval (CBIR) to exploit the information content of these images directly by using image processing. We provide an overview and current research results from this CBIR work, which includes: extensive segmentation research, focusing on Active Shape Modeling and Active Contour methods; alternative techniques for shape representation, including invariant moments, simple polygon approximation, and Fourier descriptors; neural network classification of shapes into biomedical categories, such as “anterior osteophytes present/not present”; and the implementation of a prototype CBIR system for the vertebrae that supports hybrid text/image queries using MATLAB and the MySQL relational database system.

Review of medical image retrieval systems and future directions

This paper presents a review of online systems for content-based medical image retrieval (CBIR). The objective of this review is to evaluate the capabilities and gaps in these systems and to determine ways of improving relevance of multi-modal (text and image) information retrieval in the iMedline system, being developed at the National Library of Medicine (NLM). Seven medical information retrieval systems: Figuresearch, BioText, GoldMiner, Yale Image Finder, Yottalook, Image Retrieval for Medical Applications (IRMA), and iMedline have been evaluated here using the system of gaps defined in [1]. Not all of these systems take advantage of the visual information contained in biomedical literature as figures and illustrations. However, all attempt to extract metadata about the image from the full-text of the articles and retrieve figures/images in response to a query. iMedline aims to advance the state-of-the-art in multimodal information retrieval by unifying image and text features in computing relevance. We discuss the shortcomings of these current systems and discuss future directions and next steps in iMedline toward context-based medical image retrieval.

Customized Hough transform for robust segmentation of cervical vertebrae from X-ray images

This paper addresses the issues involved in developing a robust segmentation technique capable of finding the location and orientation of the cervical vertebrae in X-ray images. This technique should be invariant to rotation, scale, noise, occlusions and shape variability. A customized approach, based on the generalized Hough transform (GHT), that captures shape variability and exploits shape information embedded in the accumulator structure to overcome noise and occlusions is proposed. This approach effectively finds estimates of the location and orientation of the cervical vertebrae boundaries in digitized X-ray images.