Pierre-Henri Conze

Objective view synthesis quality assessment

View synthesis brings geometric distortions which are not handled efficiently by existing image quality assessment metrics. Despite the widespread of 3-D technology and notably 3D television (3DTV) and free-viewpoints television (FTV), the field of view synthesis quality assessment has not yet been widely investigated and new quality metrics are required. In this study, we propose a new full-reference objective quality assessment metric: the View Synthesis Quality Assessment (VSQA) metric. Our method is dedicated to artifacts detection in synthesized view-points and aims to handle areas where disparity estimation may fail: thin objects, object borders, transparency, variations of illumination or color differences between left and right views, periodic objects... The key feature of the proposed method is the use of three visibility maps which characterize complexity in terms of textures, diversity of gradient orientations and presence of high contrast. Moreover, the VSQA metric can be defined as an extension of any existing 2D image quality assessment metric. Experimental tests have shown the effectiveness of the proposed method.

From optical flow to dense long term correspondences

Dense point matching and tracking in image sequences is an open issue with implications in several domains, from content analysis to video editing. We observe that for long term dense point matching, some regions of the image are better matched by concatenation of consecutive motion vectors, while for others a direct long term matching is preferred. We propose a method to optimally estimate the correspondence of a point w.r.t. a reference image from a set of input motion estimations over different temporal intervals. Results on texture insertion by point tracking in the context of video editing are presented and compared with a state-of-the-art approach.

Disparity-compensated view synthesis for S3D content correction

The production of stereoscopic 3D HD content is considerably increasing and experience in 2-view acquisition is in progress. High quality material to the audience is required but not always ensured, and correction of the stereo views may be required. This is done via disparity-compensated view synthesis. A robust method has been developed dealing with these acquisition problems that introduce discomfort (e.g hyperdivergence and hyperconvergence...) as well as those ones that may disrupt the correction itself (vertical disparity, color difference between views...). The method has three phases: a preprocessing in order to correct the stereo images and estimate features (e.g. disparity range...) over the sequence. The second (main) phase proceeds then to disparity estimation and view synthesis. Dual disparity estimation based on robust block-matching, discontinuity-preserving filtering, consistency and occlusion handling has been developed. Accurate view synthesis is carried out through disparity compensation. Disparity assessment has been introduced in order to detect and quantify errors. A post-processing deals with these errors as a fallback mode. The paper focuses on disparity estimation and view synthesis of HD images. Quality assessment of synthesized views on a large set of HD video data has proved the effectiveness of our method.

Robust Optical Flow Integration

We analyze the problem of how to correctly construct dense point trajectories from optical flow fields. First, we show that simple Euler integration is unavoidably inaccurate, no matter how good is the optical flow estimator. Then, an inverse integration scheme is analyzed which is more robust to bias and input noise and shows better stability properties. Our contribution is threefold: 1) a theoretical analysis that demonstrates why and in what sense inverse integration is more accurate; 2) a rich experimental validation both on synthetic and real (image) data; and 3) an algorithm for approximate online inverse integration. This new technique is precious whether one is trying to propagate information densely available on a reference frame to the other frames in the sequence or, conversely, to assign information densely over each frame by pulling it from the reference.

Multi-step flow fusion: towards accurate and dense correspondences in long video shots

The aim of this work is to estimate dense displacement fields over long video shots. Put in sequence they are useful for representing point trajectories but also for propagating (pulling) information from a reference frame to the rest of the video. Highly elaborated optical flow estimation algorithms are at hand, and they were applied before for dense point tracking by simple accumulation, however with unavoidable position drift. On the other hand, direct long-term point matching is more robust to such deviations, but it is very sensitive to ambiguous correspondences. Why not combining the benefits of both approaches? Following this idea, we develop a multi-step flow fusion method that optimally generates dense long-term displacement fields by first merging several candidate estimated paths and then filtering the tracks in the spatio-temporal domain. Our approach permits to handle small and large displacements with improved accuracy and it is able to recover a trajectory after temporary occlusions. Especially useful for video editing applications, we attack the problem of graphic element insertion and video volume segmentation, together with a number of quantitative comparisons on ground-truth data with state-of-the-art approaches.

Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans

PurposeToward an efficient clinical management of hepatocellular carcinoma (HCC), we propose a classification framework dedicated to tumor necrosis rate estimation from dynamic contrast-enhanced CT scans. Based on machine learning, it requires weak interaction efforts to segment healthy, active and necrotic liver tissues.MethodsOur contributions are two-fold. First, we apply random forest (RF) on supervoxels using multi-phase supervoxel-based features that discriminate tissues based on their dynamic in response to contrast agent injection. Second, we extend this technique in a hierarchical multi-scale fashion to deal with multiple spatial extents and appearance heterogeneity. It translates in an adaptive data sampling scheme combining RF and hierarchical multi-scale tree resulting from recursive supervoxel decomposition. By concatenating multi-phase features across the hierarchical multi-scale tree to describe leaf supervoxels, we enable RF to automatically infer the most informative scales without defining any explicit rules on how to combine them.ResultsAssessment on clinical data confirms the benefits of multi-phase information embedded in a multi-scale supervoxel representation for HCC tumor segmentation.ConclusionDedicated but not limited only to HCC management, both contributions reach further steps toward more accurate multi-label tissue classification.

Hierarchical multi-scale supervoxel matching using random forests for automatic semi-dense abdominal image registration

This paper addresses the estimation of pairwise supervoxel correspondences toward automatic semi-dense medical image registration. Supervoxel matching is performed through random forests (RF) with supervoxel indexes as label entities to predict matching areas in another target image. Ensuring accurate supervoxel boundary adherence requires a fine supervoxel decomposition which highly increases learning complexity. To alleviate this issue, we extend RF based supervoxel matching from single to multi-scale using a recursive hierarchical supervoxel representation. Output RF matching probabilities obtained for the last scale are gathered with ancestor matching probabilities which acts as a coarse-to-fine matching guidance. The effectiveness of our method is high-lighted for semi-dense abdominal image registration relying on liver label propagation and consistency assessment.

Semi-automatic Liver Tumor Segmentation in Dynamic Contrast-Enhanced CT Scans Using Random Forests and Supervoxels

Pre-operative locoregional treatments PLT delay the tumor progression by necrosis for patients with hepato-cellular carcinoma HCC. Toward an efficient evaluation of PLT response, we address the estimation of liver tumor necrosis TN from CT scans. The TN rate could shortly supplant standard criteria RECIST, mRECIST, EASL or WHO since it has recently shown higher correlation to survival rates. To overcome the inter-expert variability induced by visual qualitative assessment, we propose a semi-automatic method that requires weak interaction efforts to segment parenchyma, tumoral active and necrotic tissues. By combining SLIC supervoxels and random decision forest, it involves discriminative multi-phase cluster-wise features extracted from registered dynamic contrast-enhanced CT scans. Quantitative assessment on expert groundtruth annotations confirms the benefits of exploiting multi-phase information from semantic regions to accurately segment HCC liver tumors.

Random forests on hierarchical multi-scale supervoxels for liver tumor segmentation in dynamic contrast-enhanced CT scans

This paper addresses multi-label tissue classification in the context of liver tumor segmentation for patients with hepato-cellular carcinoma (HCC). Covering such issue in an interactive perspective through supervoxel-based random forest (RF) requires an adaptive data sampling scheme to deal with multiple spatial extents and appearance heterogeneity. We propose a simple and efficient strategy combining standard RF and hierarchical multi-scale tree resulting from recursive 3D SLIC supervoxel decomposition. By concatenating features across the hierarchical multi-scale tree to describe leaf super-voxels, we enable RF to automatically infer the most informative scales discriminating tissues based on their intrinsic properties. Our method does not require any explicit rules on how to combine the different scales. Quantitative assessment on expert ground truth annotations demonstrates improved results compared to standard single-scale strategies for HCC tumor segmentation in dynamic contrast-enhanced CT scans.

Dense motion estimation between distant frames: Combinatorial multi-step integration and statistical selection

Accurate estimation of dense point correspondences between two distant frames of a video sequence is a challenging task. To address this problem, we present a combinatorial multistep integration procedure which allows one to obtain a large set of candidate motion fields between the two distant frames by considering multiple motion paths across the video sequence. Given this large candidate set, we propose to perform the optimal motion vector selection by combining a global optimization stage with a new statistical processing. Instead of considering a selection only based on intrinsic motion field quality and spatial regularization, the statistical processing exploits the spatial distribution of candidates and introduces an intra-candidate quality based on forward-backward consistency. Experiments evaluate the effectiveness of our method for distant motion estimation in the context of video editing.