Samia Boukir

Detection and removal of line scratches in motion picture films

Line scratches are common degradations in motion picture films. This paper presents an efficient method for line scratches detection strengthened by a Kalman filter. A new interpolation technique, dealing with both low and high frequencies (i.e. film grain) around the line artifacts, is investigated to achieve a nearby invisible reconstruction of damaged areas. Our line scratches detection and removal techniques have been validated on several film sequences.

Structure from controlled motion

This paper deals with the recovery of 3D information using a single mobile camera in the context of active vision. First, we propose a general revisited formulation of the structure-from-known-motion issue. Within the same formalism, we handle various kinds of 3D geometrical primitives such as points, lines, cylinders, spheres, etc. We also aim at minimizing effects of the different measurement errors which are involved in such a process. More precisely, we mathematically determine optimal camera configurations and motions which lead to a robust and accurate estimation of the 3D structure parameters. We apply the visual servoing approach to perform these camera motions using a control law in closed-loop with respect to visual data. Real-time experiments dealing with 3D structure estimation of points and cylinders are reported. They demonstrate that this active vision strategy can very significantly improve the estimation accuracy.

Reconstruction of degraded image sequences. Application to film restoration

Abstract A suitable detection–reconstruction approach is proposed for removing impulsive distortion and other types of deterioration from degraded image sequences. The main application that has motivated this work is the problem of digital film restoration for the movie industry, which has only very recently been explored. Line artifacts, which are prominent degradations in motion picture films, are also considered here. The detection procedure consists of two steps. First, a morphological filter provides impulsive distortions and line scratch candidates. Unlike impulsive distortions, which appear randomly in an image, line artifacts persist in nearby or the same location across several frames. Furthermore, the detection process is complicated by the fact that lines occur as natural part in interesting scenes. Therefore, we add a validation step for separating possible line defects from false detections. It consists in tracking the potential line artifacts over the frames using a Kalman filter. An interpolation technique, dealing with both low and high frequencies around the detected deteriorations, is investigated to achieve a nearly invisible reconstruction of damaged areas.

Film line scratch removal using Kalman filtering and Bayesian restoration

A suitable detection/reconstruction approach is proposed for removing line scratches from degraded motion picture films. The detection procedure consists of two steps. First, a simple 1D-extrema detector provides line scratch candidates. Line artifacts persist across several frames. Therefore, to reject false detections, the detected scratches are tracked over the sequence using a Kalman filter. A new Bayesian restoration technique, dealing with both low and high frequencies around and inside the detected artifacts, is investigated to achieve a near invisible restoration of damaged areas.

Deterioration detection for digital film restoration

This paper presents a robust technique to detect local deteriorations of old cinematographic films. This method relies on spatio-temporal information and combines two different detectors: a morphological detector which uses spatial properties of deteriorations, and a dynamic detector based on motion estimation techniques. Our deterioration detector has been validated on several film sequences and turned out to be a powerful tool for digital film restoration.

Tracking and MAP Reconstruction of line scratches in degraded motion pictures

Abstract. A suitable detection and reconstruction approach is proposed for removing line scratches from degraded motion picture films. The detection procedure consists of two steps. First, a simple 1D-extrema detector provides line scratch candidates. Unlike impulsive distortions, which appear randomly in an image, line artifacts persist across several frames. Furthermore, the detection process is complicated by the fact that lines occur as a natural part in interesting scenes. Therefore, we add a validation step for separating possible line defects from false detections. It consists in tracking the potential line artifacts over the frames using a Kalman filter. A new Bayesian restoration technique, dealing with both low and high frequencies around and inside the detected deteriorations, is investigated to achieve a nearly invisible reconstruction of damaged areas.

A local method for contour matching and its parallel implementation

Abstract. This paper presents a local approach for matching contour segments in an image sequence. This study has been primarily motivated by work concerned with the recovery of 3D structure using active vision. The method to recover the 3D structure of the scene requires to track in real-time contour segments in an image sequence. Here, we propose an original and robust approach that is ideally suited for this problem. It is also of more general interest and can be used in any context requiring matching of line boundaries over time. This method only involves local modeling and computation of moving edges dealing “virtually” with a contour segment primitive representation. Such an approach brings robustness to contour segmentation instability and to occlusion, and easiness for implementation. Parallelism has also been investigated using an SIMD-based real-time image-processing system. This method has been validated with experiments on several real-image sequences. Our results show quite satisfactory performance and the algorithm runs in a few milliseconds.n

Support Vectors Selection for Supervised Learning Using an Ensemble Approach

Support Vector Machines (SVMs) are popular for pattern classification. However, training a SVM requires large memory and high processing time, especially for large datasets, which limits their applications. To speed up their training, we present a new efficient support vector selection method based on ensemble margin, a key concept in ensemble classifiers. This algorithm exploits a new version of the margin of an ensemble-based classification and selects the smallest margin instances as support vectors. Our experimental results show that our method reduces training set size significantly without degrading the performance of the resulting SVMs classifiers.

Motion compensated film restoration

Abstract. Motion picture films are susceptible to local degradations such as dust spots. Other deteriorations are global such as intensity and spatial jitter. It is obvious that motion needs to be compensated for before the detection/correction of such local and dynamic defects. Therefore, we propose a hierarchical motion estimation method ideally suited for high resolution film sequences. This recursive block-based motion estimator relies on an adaptive search strategy and Radon projections to improve processing speed. The localization of dust particles then becomes straightforward. Thus, it is achieved by simple inter-frame differences between the current image and motion compensated successive and preceding frames. However, the detection of spatial and intensity jitter requires a specific process taking advantage of the high temporal correlation in the image sequence. In this paper, we present our motion compensation-based algorithms for removing dust spots, spatial and intensity jitter in degraded motion pictures. Experimental results are presented showing the usefulness of our motion estimator for film restoration at reasonable computational costs.

Compression and recognition of dance gestures using a deformable model

In this paper, we aim for the recognition of a set of dance gestures from contemporary ballet. Our input data are motion trajectories followed by the joints of a dancing body provided by a motion-capture system. It is obvious that direct use of the original signals is unreliable and expensive. Therefore, we propose a suitable tool for non-uniform sub-sampling of spatio-temporal signals. The key to our approach is the use of a deformable model to provide a compact and efficient representation of motion trajectories. Our dance gesture recognition method involves a set of hidden Markov models (HMMs), each of them being related to a motion trajectory followed by the joints. The recognition of such movements is then achieved by matching the resulting gesture models with the input data via HMMs. We have validated our recognition system on 12 fundamental movements from contemporary ballet performed by four dancers.