Ekaterina V. Tolstaya

Content-based image orientation recognition

In this chapter, we describe a method for digital image orientation recognition. The method is based on classifier learning by a set of feature vectors extracted from images. Feature vectors are flip-invariant to effectively classify images into portrait-oriented and landscape-oriented photos. A new texture feature is proposed based on the observation that more textured areas are usually located in the lower part of the image. The proposed method could be effectively applied to index prints of photos (printing a set of miniatures of a large image collection).

Depth propagation for semi-automatic 2D to 3D conversion

In this paper, we present a method for temporal propagation of depth data that is available for so called key-frames through video sequence. Our method requires that full frame depth information is assigned. Our method utilizes nearest preceding and nearest following key-frames with known depth information. The propagation of depth information from two sides is essential as it allows to solve most occlusion problems correctly. Image matching is based on the coherency sensitive hashing (CSH) method and is done using image pyramids. Disclosed results are compared with temporal interpolation based on motion vectors from optical flow algorithm. The proposed algorithm keeps sharp depth edges of objects even in situations with fast motion or occlusions. It also handles well many situations, when the depth edges don’t perfectly correspond with true edges of objects.

Removal of blocking and ringing artifacts in JPEG-coded images

The paper relates to a method for effective reduction of artifacts, caused by lossy compression algorithms based on block-based discreet cosine transform (DCT) coding, known as JPEG coding. Most common artifacts produced by such type of coding, are blocking and ringing artifacts. To reduce the effect of coding artifacts caused by significant information loss, a variety of different algorithms and methods has been suggested. However, the majority of solutions propose to process all blocks in the image, which leads to increase of processing time, required resources, as well as image over-blurring after processing of blocks, not affected by blocking artifacts. Techniques for ringing artifact detection usually rely on edge-detection step, a complicated and versatile procedure with unknown optimal parameters. In this paper we describe very effective procedures for detection of artifacts, and their subsequent correction. This approach helps to save notable amount of computational resources, since not all the blocks are involved in correction procedures. Detection steps are performed in frequency domain, using only DCT coefficients of an image. Numerous examples have been analyzed and compared with the existent solutions, and results prove the effectiveness of proposed technique.

Fusion of high dynamic range scene photos

In this paper we propose an effective approach for creating nice-looking photo images of scenes having high dynamic range using a set of photos captured with exposure bracketing. Usually details of dark parts of the scene are preserved in over-exposed shot, and details of brightly illuminated parts are visible in under-exposed photos. A proposed method allows preservation of those details by first constructing gradient field, mapping it with special function and then integrating it to restore lightness values using Poisson equation. Resulting image can be printed or displayed on conventional displays.

Edge preserving motion estimation with occlusions correction for assisted 2D to 3D conversion

In this article we propose high quality motion estimation based on variational optical flow formulation with non-local regularization term. To improve motion in occlusion areas we introduce occlusion motion inpainting based on 3-frame motion clustering. Variational formulation of optical flow proved itself to be very successful, however a global optimization of cost function can be time consuming. To achieve acceptable computation times we adapted the algorithm that optimizes convex function in coarse-to-fine pyramid strategy and is suitable for modern GPU hardware implementation. We also introduced two simplifications of cost function that significantly decrease computation time with acceptable decrease of quality. For motion clustering based motion inpaitning in occlusion areas we introduce effective method of occlusion aware joint 3-frame motion clustering using RANSAC algorithm. Occlusion areas are inpainted by motion model taken from cluster that shows consistency in opposite direction. We tested our algorithm on Middlebury optical flow benchmark, where we scored around 20th position, but being one of the fastest method near the top. We also successfully used this algorithm in semi-automatic 2D to 3D conversion tool for spatio-temporal background inpainting, automatic adaptive key frame detection and key points tracking.

Automatic Generation of Collage

We describe algorithms for the automatic creation of collages from a collection of photos. The appeal of different forms of collage from the user’s viewpoint is discussed. We consider time- and camera-based photo selection procedures, including the estimation of the quality of the photographs. Collage generation involves arrangement of the photos on canvas and an application of seamless blending with elements of randomness.

Automatic Red Eye Correction

The red eye artefact is an irritating defect in photos. The correction of red eyes during printing without user intervention is an important task. This chapter is devoted to a description of an efficient technique for automatic correction of red eyes. Initially we developed a method for a photo printer; however, our approach is applicable for any software and firmware. The algorithm is independent of face orientation and is capable of detecting paired red eyes as well as single red eyes. For the segmentation of roundish red regions, we applied colour information and thresholding in the domain of outcomes of four-directional edge-detection filters jointly. For classification of segmented regions, we built a cascade of classifiers: three simple classifiers eliminate obviously false areas, and after that an ensemble of decision trees created by an adaptive boosting algorithm performs detection of red-eye regions with good performance. A retouching stage includes desaturation, darkening, and blending with the initial image. In addition, we construct a sophisticated quality criterion of correction: we employ the Analytic Hierarchy Process for prioritization of the observer’s opinions about outcomes of detection and correction. The experimental results demonstrate good performance of the proposed algorithm in comparison with existing solutions.

Removal of JPEG Artefacts

The present chapter generally relates to a method for effective reduction of artefacts caused by lossy compression algorithms based on block-based Discreet Cosine Transform (DCT) coding, known as JPEG coding. The most common artefacts produced by this type of coding are blocking and ringing artefacts. To reduce the effect of coding artefacts caused by significant information loss, a variety of different algorithms and methods have been suggested. However, the majority of solutions propose to process all blocks in the image, even those blocks that are not affected by artefacts and this leads to an increase in processing time and required resources, as well as image over-blurring. Techniques for ringing artefact detection usually rely on an edge-detection step, a complicated and versatile procedure with unknown optimal parameters. In this paper, we describe very effective procedures for the detection of artefacts and their subsequent correction. This approach helps to save a notable amount of computational resources, since not all the blocks are involved in correction procedures. Detection steps are performed in the frequency domain, using only the DCT coefficients of an image. Numerous examples have been analysed and compared with the existent solutions, and the results prove the effectiveness of the proposed technique.

Image Enhancement Pipeline Based on EXIF Metadata

The application of metadata from an EXIF-file for the estimation of the probability of defects existing in digital photos is considered. The following typical defects of photos are selected: exposure problems, noise, colour cast, blur, JPEG artefacts, and the presence of red eyes. An extensive database of photographs captured using ten different cameras was collected, and the influence of various EXIF-tags on the presence of defects analysed. An EXIF-based image enhancement pipeline is created which allows a reduction in the total time needed for automatic photo enhancement as a consequence of the estimation of the probability that several defects are present in the corrected photo. The optimisation of the image enhancement procedure is implemented by applying a specific classification based on the EXIF-tags contained in photographs, so that the number of cases for which a quality assessment is required is reduced.

Sketch for Eco-friendly Printing

Reducing toner or ink consumption is an important function of modern printing devices. It has a positive ecological and economic impact. We describe a technique for converting bitmaps from print-jobs to pleasant colour sketches. Such sketches contain significantly fewer colour dots than the initial images. The approach serves to reduce toner consumption. Our algorithm is based on the use of an edge-detection filter for mask creation and multiplication of the initial image by the mask. In order to construct the mask, we use a Difference-of-Gaussian filter for the purpose of blending the initial image with its blurred copy, where the alpha-channel is a saliency map according to the Pre-attentive Human Vision model. An estimation of the percentage of saved toner as well as a survey of observers proves the effectiveness of the proposed technique for saving toner in eco-friendly printing mode.