Ljiljana Platisa

Channelized Hotelling observers for the assessment of volumetric imaging data sets

Current clinical practice is rapidly moving in the direction of volumetric imaging. For two-dimensional (2D) images, task-based medical image quality is often assessed using numerical model observers. For three-dimensional (3D) images, however, these models have been little explored so far. In this work, first, two novel designs of a multislice channelized Hotelling observer (CHO) are proposed for the task of detecting 3D signals in 3D images. The novel designs are then compared and evaluated in a simulation study with five different CHO designs: a single-slice model, three multislice models, and a volumetric model. Four different random background statistics are considered, both gaussian (noncorrelated and correlated gaussian noise) and non-gaussian (lumpy and clustered lumpy backgrounds). Overall, the results show that the volumetric model outperforms the others, while the disparity between the models decreases for greater complexity of the detection task. Among the multislice models, the second proposed CHO could most closely approach the volumetric model, whereas the first new CHO seems to be least affected by the number of training samples.

Crack detection and inpainting for virtual restoration of paintings: The case of the Ghent Altarpiece

Digital image processing is proving to be of great help in the analysis and documentation of our vast cultural heritage. In this paper, we present a new method for the virtual restoration of digitized paintings with special attention for the Ghent Altarpiece (1432), a large polyptych panel painting of which very few digital reproductions exist. We achieve our objective by detecting and digitally removing cracks. The detection of cracks is particularly difficult because of the varying content features in different parts of the polyptych. Three new detection methods are proposed and combined in order to detect cracks of different sizes as well as varying brightness. Semi-supervised clustering based post-processing is used to remove objects falsely labelled as cracks. For the subsequent inpainting stage, a patch-based technique is applied to handle the noisy nature of the images and to increase the performance for crack removal. We demonstrate the usefulness of our method by means of a case study where the goal is to improve readability of the depiction of text in a book, present in one of the panels, in order to assist paleographers in its deciphering.

Virtual restoration of the ghent altarpiece using crack detection and inpainting

In this paper, we present a new method for virtual restoration of digitized paintings, with the special focus on the Ghent Altarpiece (1432), one of Belgiums greatest masterpieces. The goal of the work is to remove cracks from the digitized painting thereby approximating how the painting looked like before ageing for nearly 600 years and aiding art historical and palaeographical analysis. For crack detection, we employ a multiscale morphological approach, which can cope with greatly varying thickness of the cracks as well as with their varying intensities (from dark to the light ones). Due to the content of the painting (with extremely many fine details) and complex type of cracks (including inconsistent whitish clouds around them), the available inpainting methods do not provide satisfactory results on many parts of the painting. We show that patch-based methods outperform pixel-based ones, but leaving still much room for improvements in this application. We propose a new method for candidate patch selection, which can be combined with different patchbased inpainting methods to improve their performance in crack removal. The results demonstrate improved performance, with less artefacts and better preserved fine details.

The use of steerable channels for detecting asymmetrical signals with random orientations

In the optimization of medical imaging systems, there is a stringent need to shift from human observer studies to numerical observer studies, because of both cost and time limitations. Numerical models give an objective measure for the quality of displayed images for a given task and can be designed to predict the performance of medical specialists performing the same task. For the task of signal detection, the channelized Hotelling observer (CHO) has been successfully used, although several studies indicate an overefficiency of the CHO compared to human observers. One of the main causes of this overefficiency is attributed to the intrinsic uncertainty about the signal (such as its orientation) that a human observer is dealing with. Deeper knowledge of the discrepancies of the CHO and the human observer may provide extra insight in the processing of the human visual system and this knowledge can be utilized to better fine-tune medical imaging systems. In this paper, we investigate the optimal detection of asymmetrical signals with statistically known random orientation, based on joint detection and estimation theory. We derive the optimal channelized observer for this task and we show that the optimal detection in channel space requires the use of steerable channels, which are used in steerable pyramid transforms in image processing. Even though the use of CHOs for SKS tasks has not been studied so far, our findings indicate that CHO models can be further extended to incorporate intrinsic uncertainty about the signal to behave closer to humans. Experimental results are provided to illustrate these findings.

Visual quality assessment of H.264/AVC compressed laparoscopic video

The digital revolution has reached hospital operating rooms, giving rise to new opportunities such as tele-surgery and tele-collaboration. Applications such as minimally invasive and robotic surgery generate large video streams that demand gigabytes of storage and transmission capacity. While lossy data compression can offer large size reduction, high compression levels may significantly reduce image quality. In this study we assess the quality of compressed laparoscopic video using a subjective evaluation study and three objective measures. Test sequences were full High-Definition videos captures of four laparoscopic surgery procedures acquired on two camera types. Raw sequences were processed with H.264/AVC IPPP-CBR at four compression levels (19.5, 5.5, 2.8, and 1.8 Mbps). 16 non-experts and 9 laparoscopic surgeons evaluated the subjective quality and suitability for surgery (surgeons only) using Single Stimulus Continuous Quality Evaluation methodology. VQM, HDR-VDP-2, and PSNR objective measures were evaluated. The results suggest that laparoscopic video may be lossy compressed approximately 30 to 100 times (19.5 to 5.5 Mbps) without sacrificing perceived image quality, potentially enabling real-time streaming of surgical procedures even over wireless networks. Surgeons were sensitive to content but had large variances in quality scores, whereas non-experts judged all scenes similarly and over-estimated the quality of some sequences. There was high correlation between surgeons’ scores for quality and “suitability for surgery”. The objective measures had moderate to high correlation with subjective scores, especially when analyzed separately by camera type. Future studies should evaluate surgeons’ task performance to determine the clinical implications of conducting surgery with lossy compressed video.

Spatiogram features to characterize pearls in paintings

Objective characterization of jewels in paintings, especially pearls, has been a long lasting challenge for art historians. The way an artist painted pearls reflects his ability to observing nature and his knowledge of contemporary optical theory. Moreover, the painterly execution may also be considered as an individual characteristic useful in distinguishing hands. In this work, we propose a set of image analysis techniques to analyze and measure spatial characteristics of the digital images of pearls, all relying on the so called spatiogram image representation. Our experimental results demonstrate good correlation between the new metrics and the visually observed image features, and also capture the degree of realism of the visual appearance in the painting. In that sense, these results set the basis in creating a practical tool for art historical attribution and give strong motivation for further investigations in this direction.

Content-aware objective video quality assessment

Abstract. Since the end-user of video-based systems is often a human observer, prediction of user-perceived video quality (PVQ) is an important task for increasing the user satisfaction. Despite the large variety of objective video quality measures (VQMs), their lack of generalizability remains a problem. This is mainly due to the strong dependency between PVQ and video content. Although this problem is well known, few existing VQMs directly account for the influence of video content on PVQ. Recently, we proposed a method to predict PVQ by introducing relevant video content features in the computation of video distortion measures. The method is based on analyzing the level of spatiotemporal activity in the video and using those as parameters of the anthropomorphic video distortion models. We focus on the experimental evaluation of the proposed methodology based on a total of five public databases, four different objective VQMs, and 105 content related indexes. Additionally, relying on the proposed method, we introduce an approach for selecting the levels of video distortions for the purpose of subjective quality assessment studies. Our results suggest that when adequately combined with content related indexes, even very simple distortion measures (e.g., peak signal to noise ratio) are able to achieve high performance, i.e., high correlation between the VQM and the PVQ. In particular, we have found that by incorporating video content features, it is possible to increase the performance of the VQM by up to 20% relative to its noncontent-aware baseline.

Selecting stimuli parameters for video quality assessment studies based on perceptual similarity distances

This work presents a methodology to optimize the selection of multiple parameter levels of an image acquisition, degradation, or post-processing process applied to stimuli intended to be used in a subjective image or video quality assessment (QA) study. It is known that processing parameters (e.g. compression bit-rate) or technical quality measures (e.g. peak signal-to-noise ratio, PSNR) are often non-linearly related to human quality judgment, and the model of either relationship may not be known in advance. Using these approaches to select parameter levels may lead to an inaccurate estimate of the relationship between the parameter and subjective quality judgments – the system’s quality model. To overcome this, we propose a method for modeling the relationship between parameter levels and perceived quality distances using a paired comparison parameter selection procedure in which subjects judge the perceived similarity in quality. Our goal is to enable the selection of evenly sampled parameter levels within the considered quality range for use in a subjective QA study. This approach is tested on two applications: (1) selection of compression levels for laparoscopic surgery video QA study, and (2) selection of dose levels for an interventional X-ray QA study. Subjective scores, obtained from the follow-up single stimulus QA experiments conducted with expert subjects who evaluated the selected bit-rates and dose levels, were roughly equidistant in the perceptual quality space - as intended. These results suggest that a similarity judgment task can help select parameter values corresponding to desired subjective quality levels.

Digital Image Processing of The Ghent Altarpiece: Supporting the painting's study and conservation treatment

Hanging in the Saint Bavo Cathedral in Ghent, Belgium, is The Ghent Altarpiece, also known as The Adoration of the Mystic Lamb (see Figure 1). According to an inscription on the outer frames, it was painted by brothers Hubert and Jan van Eyck for Joos Vijd and?his wife Elisabeth Borluut in 1432. It is one of the most admired and influential paintings in the history of art and has given rise to many intriguing questions that have been puzzling art historians to date [11]. Moreover, the material history of the panels is very complicated. They were hidden, dismantled, moved away, stolen, and recovered during riots, fires and wars. The recovery of the panels by the U.S. Army in the Nazi hoards deep in the Altaussee salt mines has particularly marked memories. One panel was stolen in 1934 and never recovered. Besides varying conservation conditions, the panels underwent numerous restoration treatments and were even partially painted over.

Using channelized Hotelling observers to quantify temporal effects of medical liquid crystal displays on detection performance

Clinical practice is rapidly moving in the direction of volumetric imaging. Often, radiologists interpret these images in liquid crystal displays at browsing rates of 30 frames per second or higher. However, recent studies suggest that the slow response of the display can compromise image quality. In order to quantify the temporal effect of medical displays on detection performance, we investigate two designs of a multi-slice channelized Hotelling observer (msCHO) model in the task of detecting a single-slice signal in multi-slice simulated images. The design of msCHO models is inspired by simplifying assumptions about how humans observe while viewing in the stack-browsing mode. For comparison, we consider a standard CHO applied only on the slice where the signal is located, recently used in a similar study. We refer to it as a single-slice CHO (ssCHO). Overall, our results confirm previous findings that the slow response of displays degrades the detection performance of the observers. More specifically, the observed performance range of msCHO designs is higher compared to the ssCHO suggesting that the extent and rate of degradation, though significant, may be less drastic than previously estimated by the ssCHO. Especially, the difference between msCHO and ssCHO is more significant for higher browsing speeds than for slow image sequences or static images. This, together with their design criteria driven by the assumptions about humans, makes the msCHO models promising candidates for further studies aimed at building anthropomorphic observer models for the stack-mode image presentation.