Tomi Kauppi

The DIARETDB1 diabetic retinopathy database and evaluation protocol

Automatic diagnosis of diabetic retinopathy from digital fundus images has been an active research topic in the medical image processing community. The research interest is justified by the excellent potential for new products in the medical industry and significant reductions in health care costs. However, the maturity of proposed algorithms cannot be judged due to the lack of commonly accepted and representative image database with a verified ground truth and strict evaluation protocol. In this study, an evaluation methodology is proposed and an image database with ground truth is described. The database is publicly available for benchmarking diagnosis algorithms. With the proposed database and protocol, it is possible to compare different algorithms, and correspondingly, analyse their maturity for technology transfer from the research laboratories to the medical practice.

Constructing Benchmark Databases and Protocols for Medical Image Analysis: Diabetic Retinopathy

We address the performance evaluation practices for developing medical image analysis methods, in particular, how to establish and share databases of medical images with verified ground truth and solid evaluation protocols. Such databases support the development of better algorithms, execution of profound method comparisons, and, consequently, technology transfer from research laboratories to clinical practice. For this purpose, we propose a framework consisting of reusable methods and tools for the laborious task of constructing a benchmark database. We provide a software tool for medical image annotation helping to collect class label, spatial span, and experts confidence on lesions and a method to appropriately combine the manual segmentations from multiple experts. The tool and all necessary functionality for method evaluation are provided as public software packages. As a case study, we utilized the framework and tools to establish the DiaRetDB1 V2.1 database for benchmarking diabetic retinopathy detection algorithms. The database contains a set of retinal images, ground truth based on information from multiple experts, and a baseline algorithm for the detection of retinopathy lesions.

Fusion of Multiple Expert Annotations and Overall Score Selection for Medical Image Diagnosis

Two problems especially important for supervised learning and classification in medical image processing are addressed in this study: i) how to fuse medical annotations collected from several medical experts and ii) how to form an image-wise overall score for accurate and reliable automatic diagnosis. Both of the problems are addressed by applying the same receiver operating characteristic (ROC) framework which is made to correspond to the medical practise. The first problem arises from the typical need to collect the medical ground truth from several experts to understand the underlying phenomenon and to increase robustness. However, it is currently unclear how these expert opinions (annotations) should be combined for classification methods. The second problem is due to the ultimate goal of any automatic diagnosis, a patient-based (image-wise) diagnosis, which consequently must be the ultimate evaluation criterion before transferring any methods into practise. Various image processing methods provide several, e.g., spatially distinct, results, which should be combined into a single image-wise score value. We discuss and investigate these two problems in detail, propose good strategies and report experimental results on a diabetic retinopathy database verifying our findings.

Simple and Robust Optic Disc Localisation Using Colour Decorrelated Templates

Automatic analysis of digital fundus images, where optic disc extraction is an essential part, is an active research topic in retinal image analysis. A simple, fast and robust optic disc localisation method using colour decorrelated templates is proposed which results an accurate location of the optic disc in colour fundus images. In the training stage, PCA is performed on the colour image data to determine an eigenspace (colour decorrelated template space) that characterises optic disc colour. The colour decorralated optic disc templates are extracted from the training images projected to the colour decorrelated template space. In the localisation stage, template matching is applied to locate optic disc using the colour decorrelated templates after projecting an unseen fundus image to the colour decorrelated template space. The proposed method is compared with a method from the literature on DIARETDB1, a publicly available fundus image database.

A framework for constructing benchmark databases and protocols for retinopathy in medical image analysis

We address performance evaluation practices for developing medical image analysis methods, and contribute to the practice to establish and to share databases of medical images with verified ground truth and solid evaluation protocols. This helps to develop better algorithms, to perform profound method comparisons, including the state-of-the-art methods, and consequently, supports technology transfer from research laboratories to clinical practice. For this purpose, we propose a framework consisting of reusable methods and tools for the laborious task of constructing a benchmark database. We provide a medical image annotation software tool which helps to collect and store ground truth for retinopathy lesions from experts, including the fusion of spatial annotations from several experts. The tool and all necessary functionality for method evaluation are provided as a public software package. For demonstration purposes, we utilise the framework and tools to establish the DiaRetDB1 V2.1 database for benchmarking diabetic retinopathy detection algorithms. The database contains a set of retinal images, ground truth from several experts, and a strawman algorithm for the detection of retinopathy lesions.

Combining Multiple Image Segmentations by Maximizing Expert Agreement

A common characteristic of collecting the ground truth for medical images is that multiple experts provide only partially coherent manual segmentations, and in some cases, with varying confidence. As the result, there is considerable spatial variation between the expert segmentations, and for training and testing, the “true” ground truth is estimated by disambiguating (combining) the provided segments. STAPLE and its derivatives are the state-of-the-art approach for disambiguating multiple spatial segments provided by clinicians. In this work, we propose a simple yet effective procedure based on maximizing the joint agreement of experts. Our algorithm produces the optimal disambiguation by maximizing the agreement and no priors are used. In the experimental part, we generate a new ground truth for the popular diabetic retinopathy benchmark, DiaRetDB1, for which the original expert markings are publicly available. We demonstrate performance superior to the original and also STAPLE generated ground truth. In addition, the DiaRetDB1 baseline method performs better with the new ground truth.

Detection and decision-support diagnosis of diabetic retinopathy using machine vision

This invited paper considers the results of the IMAGERET project. The goal of the project is to demonstrate how lesions in a retina caused by diabetic retinopathy can be detected from color fundus images by using machine vision methods. The project consists of the following results: an image annotation tool for medical expert annotation, diabetic retinopathy databases, an evaluation framework for development and comparison of methods, image-based and pixel-based methods, and new imaging solutions. The automated diagnosis can be seen in two steps: in the first step, it is decided whether an eye needs further analysis (too many lesions visible) or not (the eye is healthy enough). In the second step, fundus images selected for further analysis are automatically diagnosed. The developed system saves both resources of medical experts and costs in healthcare. It will further offer a tool for the health care providers to improve the quality of the life of diabetes patients. This is important since the number of diabetes patients is increasing, especially rapidly in the developed countries.