Cara Calvelli

An Expert-in-the-loop Paradigm for Learning Medical Image Grouping

Image grouping in knowledge-rich domains is challenging, since domain knowledge and expertise are key to transform image pixels into meaningful content. Manually marking and annotating images is not only labor-intensive but also ineffective. Furthermore, most traditional machine learning approaches cannot bridge this gap for the absence of experts’ input. We thus present an interactive machine learning paradigm that allows experts to become an integral part of the learning process. This paradigm is designed for automatically computing and quantifying interpretable grouping of dermatological images. In this way, the computational evolution of an image grouping model, its visualization, and expert interactions form a loop to improve image grouping. In our paradigm, dermatologists encode their domain knowledge about the medical images by grouping a small subset of images via a carefully designed interface. Our learning algorithm automatically incorporates these manually specified connections as constraints for re-organizing the whole image dataset. Performance evaluation shows that this paradigm effectively improves image grouping based on expert knowledge.

Human-centric approaches to image understanding and retrieval

The amount of digital medical image data is increasing rapidly in terms of both quantity and heterogeneity. There exists a great need to format medical image archives so as to facilitate diagnostics and preventive medicine. To achieve this, in the past few decades great efforts have been made to investigate methods of applying content-based image retrieval (CBIR) techniques to retrieve images. However, several critical challenges remain. Recently, CBIR research has become intertwined with the fundamental problem of image understanding and it is recognized that computing solutions that bridge the “semantic gap” must capture higher-level domain knowledge of medical end users. We are investigating the incorporation of state-of-the-art visual categorization techniques into conventional CBIR approaches. Visual attention deployment strategies of medical experts serve as an objective measure to help us understand the perceptual and conceptual processes involved in identifying key visual features and selecting diagnostic regions of the images. Understanding these processes will inform and direct feature selection approaches on medical images, such as the dermatological images used in our study. We also explore systematic and effective information integration methods of image data and semantic descriptions with the long-term goals of building efficient human-centered multi-modal interactive CBIR systems.

Intelligent medical image grouping through interactive learning

Image grouping in knowledge-rich domains is challenging, since domain knowledge and human expertise are key to transform image pixels into meaningful content. Manually marking and annotating images is not only labor-intensive but also ineffective. Furthermore, most traditional machine learning approaches cannot bridge this gap for the absence of experts’ input. We thus present an interactive machine learning paradigm that allows experts to become an integral part of the learning process. This paradigm is designed for automatically computing and quantifying interpretable grouping of dermatological images. In this way, the computational evolution of an image grouping model, its visualization, and expert interactions form a loop to improve image grouping. In our paradigm, dermatologists encode their domain knowledge about the medical images by grouping a small subset of images via a carefully designed interface. Our learning algorithm automatically incorporates these manually specified connections as constraints for reorganizing the whole image dataset. Performance evaluation shows that this paradigm effectively improves image grouping based on expert knowledge.