Ferran Marqués

Multiscale Combinatorial Grouping

We propose a unified approach for bottom-up hierarchical image segmentation and object candidate generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object candidates by exploring efficiently their combinatorial space. We conduct extensive experiments on both the BSDS500 and on the PASCAL 2012 segmentation datasets, showing that MCG produces state-of-the-art contours, hierarchical regions and object candidates.

Region-based representations of image and video: segmentation tools for multimedia services

This paper discusses region-based representations of image and video that are useful for multimedia services such as those supported by the MPEG-4 and MPEG-7 standards. Classical tools related to the generation of the region-based representations are discussed. After a description of the main processing steps and the corresponding choices in terms of feature spaces, decision spaces, and decision algorithms, the state of the art in segmentation is reviewed. Mainly tools useful in the context of the MPEG-4 and MPEG-7 standards are discussed. The review is structured around the strategies used by the algorithms (transition based or homogeneity based) and the decision spaces (spatial, spatio-temporal, and temporal). The second part of this paper proposes a partition tree representation of images and introduces a processing strategy that involves a similarity estimation step followed by a partition creation step. This strategy tries to find a compromise between what can be done in a systematic and universal way and what has to be application dependent. It is shown in particular how a single partition tree created with an extremely simple similarity feature can support a large number of segmentation applications: spatial segmentation, motion estimation, region-based coding, semantic object extraction, and region-based retrieval.

Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation

We propose a unified approach for bottom-up hierarchical image segmentation and object proposal generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object proposals by exploring efficiently their combinatorial space. We also present Single-scale Combinatorial Grouping (SCG), a faster version of MCG that produces competitive proposals in under five seconds per image. We conduct an extensive and comprehensive empirical validation on the BSDS500, SegVOC12, SBD, and COCO datasets, showing that MCG produces state-of-the-art contours, hierarchical regions, and object proposals.

Segmentation-based video coding system allowing the manipulation of objects

This paper presents a generic video coding algorithm allowing the content-based manipulation of objects. This manipulation is possible thanks to the definition of a spatiotemporal segmentation of the sequences. The coding strategy relies on a joint optimization in the rate-distortion sense of the partition definition and of the coding techniques to be used within each region. This optimization creates the link between the analysis and synthesis parts of the coder. The analysis defines the time evolution of the partition, as well as the elimination or the appearance of regions that are homogeneous either spatially or in motion. The coding of the texture as well as of the partition relies on region-based motion compensation techniques. The algorithm offers a good compromise between the ability to track and manipulate objects and the coding efficiency.

Region Merging Techniques Using Information Theory Statistical Measures

The purpose of the current work is to propose, under a statistical framework, a family of unsupervised region merging techniques providing a set of the most relevant region-based explanations of an image at different levels of analysis. These techniques are characterized by general and nonparametric region models, with neither color nor texture homogeneity assumptions, and a set of innovative merging criteria, based on information theory statistical measures. The scale consistency of the partitions is assured through i) a size regularization term into the merging criteria and a classical merging order, or ii) using a novel scale-based merging order to avoid the region size homogeneity imposed by the use of a size regularization term. Moreover, a partition significance index is defined to automatically determine the subset of most representative partitions from the created hierarchy. Most significant automatically extracted partitions show the ability to represent the semantic content of the image from a human point of view. Finally, a complete and exhaustive evaluation of the proposed techniques is performed, using not only different databases for the two main addressed problems (object-oriented segmentation of generic images and texture image segmentation), but also specific evaluation features in each case: under- and oversegmentation error, and a large set of region-based, pixel-based and error consistency indicators, respectively. Results are promising, outperforming in most indicators both object-oriented and texture state-of-the-art segmentation techniques.

Binary Partition Trees for Object Detection

This paper discusses the use of binary partition trees (BPTs) for object detection. BPTs are hierarchical region-based representations of images. They define a reduced set of regions that covers the image support and that spans various levels of resolution. They are attractive for object detection as they tremendously reduce the search space. In this paper, several issues related to the use of BPT for object detection are studied. Concerning the tree construction, we analyze the compromise between computational complexity reduction and accuracy. This will lead us to define two parts in the BPT: one providing accuracy and one representing the search space for the object detection task. Then we analyze and objectively compare various similarity measures for the tree construction. We conclude that different similarity criteria should be used for the part providing accuracy in the BPT and for the part defining the search space and specific criteria are proposed for each case. Then we discuss the object detection strategy based on BPT. The notion of node extension is proposed and discussed. Finally, several object detection examples illustrating the generality of the approach and its efficiency are reported.

Automatic satellite image georeferencing using a contour-matching approach

Multitemporal and multisatellite studies or comparisons between satellite data and local ground measurements require nowadays precise and automatic geometric correction of satellite images. This paper presents a fully automatic geometric correction system capable of georeferencing satellite images with high accuracy. An orbital prediction model, which provides initial earth locations, is combined with the proposed automatic contour-matching technique. This combination allows correcting the low-frequency error component, mainly due to timing and orbital model errors, as well as the high-frequency error component, due to variations in the spacecrafts attitude. The approach aims at exploiting the maximum reliable information in the image to guide the matching algorithm. The contour-matching process has three main steps: 1) estimation of the gradient energy map (edges) and detection of the cloudless (reliable) areas; 2) initialization of the contours positions; 3) estimation of the transformation parameters (affine model) using a contour optimization approach. Three different robust and automatic algorithms are proposed for optimization, and their main features are discussed. Finally, the performance of the three proposed algorithms is assessed using a new error estimation technique applied to Advanced Very High Resolution Radiometer (AVHRR), Sea-viewing Wide Field of view Sensor (SeaWiFS), and multisensor AVHRR-SeaWiFS imagery.

Automatic tool for the precise detection of upwelling and filaments in remote sensing imagery

The upward movement of cool and nutrient-rich waters toward the surface leads to horizontal alterations in the distribution of the physical, chemical, and biological properties. Remote sensing is being extensively applied to detect such coastal upwellings; however, the enormous amount of data daily generated obliges to develop automatic detection and prediction tools. The problem of identifying oceanographic mesoscale structures has been studied using a variety of image processing techniques; however, the outstanding difficulties encountered in the traditional approaches are the presence of noise, the fact that gradients are weak, the strong morphological variation, and the absence of a valid analytical model for the structures. In this context, the proposed automatic upwelling extraction methodology overcomes the preceding detection inconveniences and achieves a highly accurate structure extraction. This automatic technique is based on a coarse-segmentation methodology followed by a fine-detail growing process. The complete system has been validated over a database of 378 multisensorial images of years 2000 to 2003, and it has been applied to the detection and feature extraction of coastal upwellings and filaments in three areas with different characteristics, such as the Canary Islands, Cape Ghir, and the Alboran Sea, using imagery from the Advanced Very High Resolution Radiometer 2 and 3 sensors, the Sea-viewing Wide Field-of-view Sensor, and the Moderate Resolution Imaging Spectroradiometer sensor, demonstrating its effectiveness and robustness in a wide variety of climate conditions.

Tracking of generic objects for video object generation

A tracking technique for creating generic video objects is presented. It assumes that the object in the initial image has been previously defined by an object partition. The object tracking relies on the concept of partition projection. The projection of a partition accommodates the previous partition information into the current image. The object partition is re-segmented so a texture partition is created. The texture partition guarantees the spatial homogeneity of each region and it is projected on the following image using motion information. Projected regions are used as markers and, to validate the position of the markers, they are fit into an intra fine partition of the current image. The texture partition of the current image is obtained by a modified watershed algorithm based on the fit markers. Finally, the current object partition is obtained by a labeling process.

Automatic registration of 3D MR images with a computerized brain atlas.

We present an automatic and unsupervised method for non-rigid registration of 3D magnetic resonance (MR) images with the Stockholm Computerized Brain Atlas (CBA). This method can be used in the context of multimodal medical image registration, fusion and automatic brain segmentation. In these applications anatomical images (MR) are coregistered with low spatial resolution functional imaging modalities (PET and SPECT) and fused with the neurological database of the CBA. The proposed matching method is based on the minimization of a 3D Chamfer distance function between the surface of the brain extracted from the MR image and the CBA brain surface. The surface-to-surface distance function is efficiently calculated by using a precomputed point-to-surface Euclidean distance map. The non-rigid inter-patient transformation of the CBA is modeled by a generalized 3D second order transformation. This transformation is easily differentiable and, as a consequence, fast and efficient minimization methods can be used. First, a quasi-rigid, first order transformation is computed. Then, the matching is improved by introducing the second order coefficients into the transformation. After this global matching, a local adaptation of the CBA is performed by a morphing method. The combination of a second order global transformation with a 3D local morphing allows the user to obtain a registration accuracy of one pixel, i.e. a mean distance between the surface of the brain in the MR image and the CBA of one pixel, which is significantly better than what can be expected from a human operator.