Andrei Rares

Edge-based image restoration

In this paper, we propose a new image inpainting algorithm that relies on explicit edge information. The edge information is used both for the reconstruction of a skeleton image structure in the missing areas, as well as for guiding the interpolation that follows. The structure reconstruction part exploits different properties of the edges, such as the colors of the objects they separate, an estimate of how well one edge continues into another one, and the spatial order of the edges with respect to each other. In order to preserve both sharp and smooth edges, the areas delimited by the recovered structure are interpolated independently, and the process is guided by the direction of the nearby edges. The novelty of our approach lies primarily in exploiting explicitly the constraint enforced by the numerical interpretation of the sequential order of edges, as well as in the pixel filling method which takes into account the proximity and direction of edges. Extensive experiments are carried out in order to validate and compare the algorithm both quantitatively and qualitatively. They show the advantages of our algorithm and its readily application to real world cases.

Complex event classification in degraded image sequences

We address the problem of motion estimation failure in degraded image sequences. This failure is caused by some complex events that take place in the image. As a result, the sequence of operations that rely on the motion estimation process, such as motion compensation and motion picture restoration, fail as well. The statistical analysis of the complex event areas indicates that it is possible to discriminate between the complex events resulting from complicated object motion and the ones resulting from image artefacts. An analysis scheme based on segment matching is proposed for the task of classifying the detected complex event areas.

A spatiotemporal image sequence restoration algorithm

Temporal restoration of image sequences may fail because of erroneously estimated motion. Motion estimation failures may occur for several reasons -complicated motion, severe artefacts, or a combination of both. We present an algorithm that tries to overcome at least the problems related to severe artefacts. It consists of a spatial restoration, which is supposed to recover the general image structure within the missing areas, followed by a temporal restoration. Performing spatial restoration first has the advantage that motion estimation during temporal restoration can rely on the spatially restored frames. The efficiency of the spatial restoration scheme is demonstrated for both artificial and real life examples. Restoration results when using only temporal restoration are presented for the real life sequences. Finally, we compare both restoration methods with the results of the proposed spatiotemporal algorithm and show its superior behaviour.

Object tracking by adaptive modeling

This paper addresses the problem of object tracking in image sequences. The approach taken is based upon adaptive statistical models. An object selected in a frame by a user is tracked throughout the sequence by using a blob-like description of its features. The object features are continuously updated by using the on-line version of the expectation-maximization algorithm. The proposed object description results in a flexible representation.

Image sequence restoration in the presence of pathological motion and severe artifacts

Incorrect motion vectors represent the main reason why current image sequence restoration schemes fail. We present a scheme that 1) identifies areas that are likely to contain wrong motion vectors, 2) finds artifacts within these areas, and 3) restores these artifacts: Although temporal information is commonly used in nowadays restoration systems [7, 12], in this particular case the artifact restoration cannot rely on it due to the uncertain motion information. Hence, the restoration needs to rely on spatial information alone. The novel spatial restoration algorithm that we introduce here, performs a non-linear interpolation that preserves the edges surrounding the artifact area. In this way, the general structure of the image is reconstructed. The restoration algorithm is demonstrated on both artificial and real life examples, and the advantages of the proposed edge-based restoration are highlighted. In addition, results are shown for the proposed complete image sequence restoration scheme.

3.11 – Video Enhancement and Restoration

Even with the advancing camera and digital recording technology, there are many situations in which recorded image sequences ‐ or video for short ‐may suffer from severe degradations. The poor quality of recorded image sequences may be due to, for instance, the imperfect or uncontrollable recording conditions such as one encounters in astronomy, forensic sciences, and medical imaging. Video enhancement and restoration has always been important in these application areas not only to improve the visual quality, but also to increase the performance of subsequent tasks such as analysis and interpretation. Another important application of video enhancement and restoration is that of preserving motion pictures and video tapes recorded over the last century. These unique records of historic, artistic, and cultural developments are deteriorating rapidly due to aging effects of the physical reels of film and magnetic tapes that carry the information. The preservation of these fragile archives is of interest not only to professional archivists, but also to broadcasters as a cheap alternative to fill the many television channels that have come available with digital broadcasting. Re-using old film and video material is, however, only feasible if the visual quality meets the standards of today. First, the archived film and video is transferred from the original film reels or magnetic tape to digital media. Then, all kinds of degradations are removed from the digitized image sequences, in this way increasing the visual quality and commercial value. Because the objective of restoration is to remove irrelevant information such as noise and edges, it restores the original spatial and temporal correlation structure of digital image sequences. Consequently, restoration may also improve the efficiency of the subsequent MPEG compression of image sequences. An important difference between the enhancement and restoration of 2-D images and of video is the amount of data to be processed. Whereas for the quality improvement of important images elaborate processing is still feasible, this is no longer true for the absolutely huge amounts of pictorial information encountered in medical sequences and film/video archives. Consequently, enhancement and restoration methods for image sequences should be fit for ‐ at least partial ‐ implementation in hardware, should have a manageable complexity, and should be semiautomatic. The term semi-automatic indicates that in the end professional operators control the visual quality of the restored image sequences by selecting values for some of the critical restoration parameters. The most common artifact encountered in the above-mentioned applications is noise. Over the last two decades an enormous amount of research has focused on the problem of enhancing and restoring 2-D images. Clearly, the resulting spatial methods are also applicable to image sequences, but such an approach implicitly assumes that the individual pictures of the image sequence, or frames, are temporally independent. By ignoring the temporal correlation that

Constrained texture restoration

A method is proposed for filling in missing areas of degraded images through explicit structure reconstruction, followed by texture synthesis. The structure being reconstructed represents meaningful edges from the image, which are traced inside the artefact. The structure reconstruction step relies on different properties of the edges touching the artefact and of the areas between them, in order to sketch the missing edges within the artefact area. The texture synthesis step is based on Markov random fields and is constrained by the traced edges in order to preserve both the shape and the appearance of the various regions in the image. The novelty of our contribution concerns constraining the texture synthesis, which proves to give results superior to the original texture synthesis alone, or to the smoothness-preserving structure-based restoration.