Adrien Depeursinge

Fusion Techniques for Combining Textual and Visual Information Retrieval

This chapter describes several approaches for information fusion that have been used in ImageCLEF over the past seven years. In this context, the fusion of information is mainly meant to combine textual and visual retrieval. Data fusion techniques from 116 papers (62% of ImageCLEF working notes) are categorized, described and discussed. It was observed that three general approaches were used for retrieval that can be categorized based on the system level chosen for combining modalities: 1) at the input of the system with inter–media query expansion, 2) internally to the system with early fusion and 3) at the output of the system with late fusion which is by far the most widely used fusion strategy.

Information Fusion for Combining Visual and Textual Image Retrieval

In this paper, classical approaches such as maximum combinations (combMAX), sum combinations (comb-SUM) and the product of the maximum and a non–zero number (combMNZ) were employed and the trade–off between two fusion effects (chorus and dark horse effects) was studied based on the sum of n maximums. Various normalization strategies were tried out. The fusion algorithms are evaluated using the best four visual and textual runs of the ImageCLEF medical image retrieval task 2008 and 2009. The results show that fused runs outperform the best original runs and multi-modality fusion statistically outperforms single modality fusion. The logarithmic rank penalization shows to be the most stable normalization. The dark horse effect is in competition with the chorus effect and each of them can produce best fusion performance depending on the nature of the input data.

A classification framework for lung tissue categorization

We compare five common classifier families in their ability to categorize six lung tissue patterns in high-resolution computed tomography (HRCT) images of patients affected with interstitial lung diseases (ILD) but also normal tissue. The evaluated classifiers are Naive Bayes, k-Nearest Neighbor (k-NN), J48 decision trees, Multi-Layer Perceptron (MLP) and Support Vector Machines (SVM). The dataset used contains 843 regions of interest (ROI) of healthy and five pathologic lung tissue patterns identified by two radiologists at the University Hospitals of Geneva. Correlation of the feature space composed of 39 texture attributes is studied. A grid search for optimal parameters is carried out for each classifier family. Two complementary metrics are used to characterize the performances of classification. Those are based on McNemars statistical tests and global accuracy. SVM reached best values for each metric and allowed a mean correct prediction rate of 87.9% with high class-specific precision on testing sets of 423 ROIs.

Image-based diagnostic aid for interstitial lung disease with secondary data integration

Interstitial lung diseases (ILDs) are a relatively heterogeneous group of around 150 illnesses with often very unspecific symptoms. The most complete imaging method for the characterisation of ILDs is the high-resolution computed tomography (HRCT) of the chest but a correct interpretation of these images is difficult even for specialists as many diseases are rare and thus little experience exists. Moreover, interpreting HRCT images requires knowledge of the context defined by clinical data of the studied case. A computerised diagnostic aid tool based on HRCT images with associated medical data to retrieve similar cases of ILDs from a dedicated database can bring quick and precious information for example for emergency radiologists. The experience from a pilot project highlighted the need for detailed database containing high-quality annotations in addition to clinical data. The state of the art is studied to identify requirements for image-based diagnostic aid for interstitial lung disease with secondary data integration. The data acquisition steps are detailed. The selection of the most relevant clinical parameters is done in collaboration with lung specialists from current literature, along with knowledge bases of computer-based diagnostic decision support systems. In order to perform high-quality annotations of the interstitial lung tissue in the HRCT images an annotation software and its own file format is implemented for DICOM images. A multimedia database is implemented to store ILD cases with clinical data and annotated image series. Cases from the University & University Hospitals of Geneva (HUG) are retrospectively and prospectively collected to populate the database. Currently, 59 cases with certified diagnosis and their clinical parameters are stored in the database as well as 254 image series of which 26 have their regions of interest annotated. The available data was used to test primary visual features for the classification of lung tissue patterns. These features show good discriminative properties for the separation of five classes of visual observations.

Learning a Frequency---Based Weighting for Medical Image Classification

This article describes the use of a frequency---based weighting developed for image retrieval to perform automatic annotation of images (medical and non---medical). The techniques applied are based on a simple tf/idf(term frequency, inverse document frequency) weighting scheme of GIFT (GNU Image Finding Tool), which is augmented by feature weights extracted from training data. The additional weights represent a measure of discrimination by taking into account the number of occurrences of the features in pairs of images of the same class or in pairs of images from different classes. The approach is fit to the image classification task by pruning parts of the training data. Further investigations were performed showing that weightings lead to significantly worse classification quality in certain feature domains. A classifier using a mixture of tf/idfweighted scoring, learned feature weights, and regular Euclidean distance gave best results using only the simple features. Using the aspect---ratio of images as feature improved results significantly.