Tijana Ruzic

Context-Aware Patch-Based Image Inpainting Using Markov Random Field Modeling

In this paper, we first introduce a general approach for context-aware patch-based image inpainting, where textural descriptors are used to guide and accelerate the search for well-matching (candidate) patches. A novel top-down splitting procedure divides the image into variable size blocks according to their context, constraining thereby the search for candidate patches to nonlocal image regions with matching context. This approach can be employed to improve the speed and performance of virtually any (patch-based) inpainting method. We apply this approach to the so-called global image inpainting with the Markov random field (MRF) prior, where MRF encodes a priori knowledge about consistency of neighboring image patches. We solve the resulting optimization problem with an efficient low-complexity inference method. Experimental results demonstrate the potential of the proposed approach in inpainting applications like scratch, text, and object removal. Improvement and significant acceleration of a related global MRF-based inpainting method is also evident.

Markov Random Field based image inpainting with context-aware label selection

In this paper, we propose a novel global Markov Random Field based image inpainting method with context-aware label selection. Context is determined based on the texture and color features in fixed image regions and is used to distinguish areas of similar content to which the search for candidate patches is limited. Furthermore, we introduce a novel optimization approach, as an alternative to priority belief propagation framework, which further reduces the number of candidates and performs efficient inference to obtain final inpainting result. Experimental results show improvement over related state-of-the-art methods. Moreover, global optimization is significantly accelerated with the proposed inference approach.

Spatiogram features to characterize pearls in paintings

Objective characterization of jewels in paintings, especially pearls, has been a long lasting challenge for art historians. The way an artist painted pearls reflects his ability to observing nature and his knowledge of contemporary optical theory. Moreover, the painterly execution may also be considered as an individual characteristic useful in distinguishing hands. In this work, we propose a set of image analysis techniques to analyze and measure spatial characteristics of the digital images of pearls, all relying on the so called spatiogram image representation. Our experimental results demonstrate good correlation between the new metrics and the visually observed image features, and also capture the degree of realism of the visual appearance in the painting. In that sense, these results set the basis in creating a practical tool for art historical attribution and give strong motivation for further investigations in this direction.

Digital Image Processing of The Ghent Altarpiece: Supporting the painting's study and conservation treatment

Hanging in the Saint Bavo Cathedral in Ghent, Belgium, is The Ghent Altarpiece, also known as The Adoration of the Mystic Lamb (see Figure 1). According to an inscription on the outer frames, it was painted by brothers Hubert and Jan van Eyck for Joos Vijd and?his wife Elisabeth Borluut in 1432. It is one of the most admired and influential paintings in the history of art and has given rise to many intriguing questions that have been puzzling art historians to date [11]. Moreover, the material history of the panels is very complicated. They were hidden, dismantled, moved away, stolen, and recovered during riots, fires and wars. The recovery of the panels by the U.S. Army in the Nazi hoards deep in the Altaussee salt mines has particularly marked memories. One panel was stolen in 1934 and never recovered. Besides varying conservation conditions, the panels underwent numerous restoration treatments and were even partially painted over.

Single-image super-resolution using sparsity constraints and non-local similarities at multiple resolution scales

Traditional super-resolution methods produce a clean high-resolution image from several observed degraded low-resolution images following an acquisition or degradation model. Such a model describes how each output pixel is related to one or more input pixels and it is called data fidelity term in the regularization framework. Additionally, prior knowledge such as piecewise smoothness can be incorporated to improve the image restoration result. The impact of an observed pixel on the restored pixels is thus local according to the degradation model and the prior knowledge. Therefore, the traditional methods only exploit the spatial redundancy in a local neighborhood and are therefore referred to as local methods. Recently, non-local methods, which make use of similarities between image patches across the whole image, have gained popularity in image restoration in general. In super-resolution literature they are often referred to as exemplarbased methods. In this paper, we exploit the similarity of patches within the same scale (which is related to the class of non-local methods) and across different resolution scales of the same image (which is also related to the fractal-based methods). For patch fusion, we employ a kernel regression algorithm, which yields a blurry and noisy version of the desired high-resolution image. For the final reconstruction step, we develop a novel restoration algorithm. The joint deconvolution/denoising algorithm is based on the split Bregman iterations and, as prior knowledge, the algorithm exploits the sparsity of the image in the shearlet-transformed domain. Initial results indicate an improvement over both classical local and state-of-the art non-local super-resolution methods.

Depth-guided patch-based disocclusion filling for view synthesis via Markov random field modelling

In this paper, we propose a novel patch-based disocclusion filling method for view synthesis from video-plus-depth data. The proposed method treats disocclusion filling as a global optimization problem, where global (spatial) consistency among the patches is enforced via a Markov random field (MRF) model. The main idea of our method is to exploit disocclusion properties to limit and guide the search for candidate patches (labels) and to minimize efficiently the resulting MRF energy. In particular, we propose to constrain the label selection to local background regions in order to ensure that the disocclusions are filled with background information. Background is determined based on the locally estimated hard threshold on the depth values. The efficient minimization approach represents an extension of our previous method for general inpainting, where we propose to visit the MRF nodes from the background to the foreground disocclusion border and discard unnecessary labels. In this way, the number of labels is further reduced and the propagation of background information is additionally enforced. Finally, efficient inference is performed to obtain the final inpainting result. The proposed disocclusion filling method represents one step of the complete view synthesis framework that we also introduce in this paper. Experimental results show improvement of the proposed approach over related state-of-the-art methods both for small and big disocclusions.

Exploring contour and texture features for context-aware patch-based inpainting

In this paper, we explore the use of contour and texture features for context-aware patch-based image inpainting. Both of these features are obtained by analysing the image filtered with the bank of filters at multiple orientations and scales, specifically Gabor filters. We use contour features to define a novel patch priority, which represents the main contribution of this paper. The priority is used to determine the filling order of the missing region, which is crucial for the success of the algorithm. Our goal is to make better differentiation between patches with structured, textured and smooth content than related definitions. We employ this novel priority within our recently proposed context-aware inpainting method, which uses contextual descriptors to find contextually similar image regions to which the search for well matching replacement patches is constrained. Here we use texture features, together with color features, as contextual descriptors of image regions. The benefit of the context-aware approach is twofold: the chance of choosing wrong matches is reduced and the search for candidate patches is accelerated. Experimental results demonstrate the benefit of the proposed method compared to state-of-the-art.