Luís Corte-Real

Toward a generic evaluation of image segmentation

Image segmentation plays a major role in a broad range of applications. Evaluating the adequacy of a segmentation algorithm for a given application is a requisite both to allow the appropriate selection of segmentation algorithms as well as to tune their parameters for optimal performance. However, objective segmentation quality evaluation is far from being a solved problem. In this paper, a generic framework for segmentation evaluation is introduced after a brief review of previous work. A metric based on the distance between segmentation partitions is proposed to overcome some of the limitations of existing approaches. Symmetric and asymmetric distance metric alternatives are presented to meet the specificities of a wide class of applications. Experimental results confirm the potential of the proposed measures.

Automatic Image Registration Through Image Segmentation and SIFT

Automatic image registration (AIR) is still a present challenge for the remote sensing community. Although a wide variety of AIR methods have been proposed in the last few years, there are several drawbacks which avoid their common use in practice. The recently proposed scale invariant feature transform (SIFT) approach has already revealed to be a powerful tool for the obtention of tie points in general image processing tasks, but it has a limited performance when directly applied to remote sensing images. In this paper, a new AIR method is proposed, based on the combination of image segmentation and SIFT, complemented by a robust procedure of outlier removal. This combination allows for an accurate obtention of tie points for a pair of remote sensing images, being a powerful scheme for AIR. Both synthetic and real data have been considered in this work for the evaluation of the proposed methodology, comprising medium and high spatial resolution images, and single-band, multispectral, and hyperspectral images. A set of measures which allow for an objective evaluation of the geometric correction process quality has been used. The proposed methodology allows for a fully automatic registration of pairs of remote sensing images, leading to a subpixel accuracy for the whole considered data set. Furthermore, it is able to account for differences in spectral content, rotation, scale, translation, different viewpoint, and change in illumination.

Measures for an Objective Evaluation of the Geometric Correction Process Quality

The geometric correction process is a crucial step in remote sensing applications. This process is frequently manually performed-which is a laborious task in many situations-as automatic image registration methods are still far from being broadly applied. One of the reasons that justify the absence of a broad application of automatic image registration methods is the lack of measures for an objective and automated analysis of the image registration process quality. The root mean square (RMS) of the residuals is the only quantitative evaluation which is generally used in this process, with the final validation of the geometric correction process being a qualitative analysis. Therefore, in both ldquohumanrdquo and automatic image registration processes, an objective evaluation of its quality is required. In this letter, we propose several measures for an objective evaluation of the geometric correction process, as a complement to the traditional RMS of the residuals and visual inspection. Two scenarios of control point distribution and the most common residual distributions were considered. With the proposed measures, we intend to cover the most common qualitative analysis aspects. This has particular importance under the scope of automatic image registration methods, where an automatic evaluation of the results is also required.

Video object matching across multiple independent views using local descriptors and adaptive learning

Object detection and tracking is an essential preliminary task in event analysis systems (e.g. visual surveillance). Typically objects are extracted and tagged, forming representative tracks of their activity. Tagging is usually performed by probabilistic data association, however, in systems capturing disjoint areas it is often not possible to establish such associations, as data may have been collected at different times or in different locations. In this case, appearance matching is a valuable aid. We propose using bag-of-visterms, i.e. an histogram of quantized local feature descriptors, to represent and match tracked objects. This method has proven to be effective for object matching and classification in image retrieval applications, where descriptors can be extracted a priori. An important difference in event analysis systems is that relevant information is typically restricted to the foreground. Descriptors can, therefore, be extracted faster, approaching real-time requirements. Also, unlike image retrieval, objects can change over time and therefore their model needs to be updated continuously. Incremental or adaptive learning is used to tackle this problem. Using independent tracks of 30 different persons, we show that the bag-of-visterms representation effectively discriminates visual object tracks and that it presents high resilience to incorrect object segmentation. Additionally, this methodology allows the construction of scalable object models that can be used to match tracks across independent views.

HAIRIS: A Method for Automatic Image Registration Through Histogram-Based Image Segmentation

Automatic image registration is still an actual challenge in several fields. Although several methods for automatic image registration have been proposed in the last few years, it is still far from a broad use in several applications, such as in remote sensing. In this paper, a method for automatic image registration through histogram-based image segmentation (HAIRIS) is proposed. This new approach mainly consists in combining several segmentations of the pair of images to be registered, according to a relaxation parameter on the histogram modes delineation (which itself is a new approach), followed by a consistent characterization of the extracted objects-through the objects area, ratio between the axis of the adjust ellipse, perimeter and fractal dimension-and a robust statistical based procedure for objects matching. The application of the proposed methodology is illustrated to simulated rotation and translation. The first dataset consists in a photograph and a rotated and shifted version of the same photograph, with different levels of added noise. It was also applied to a pair of satellite images with different spectral content and simulated translation, and to real remote sensing examples comprising different viewing angles, different acquisition dates and different sensors. An accuracy below 1° for rotation and at the subpixel level for translation were obtained, for the most part of the considered situations. HAIRIS allows for the registration of pairs of images (multitemporal and multisensor) with differences in rotation and translation, with small differences in the spectral content, leading to a subpixel accuracy.

A very low bit rate video coder based on vector quantization

Describes a video coder based on a hybrid DPCM-vector quantization algorithm that is suited for bit rates ranging from 8-16 kb/s. The proposed approach involves segmenting difference images into variable-size and variable-shape blocks and performing segmentation and motion compensation simultaneously. The purpose of obtaining motion vectors for variable-size and variable-shape blocks is to improve the quality of motion estimation, particularly in those areas where the edges of moving objects are situated. For the larger blocks, decimation takes place in order to simplify vector quantization. For very active blocks, which are always of small dimension, a specific vector quantizer has been applied, the fuzzy classified vector quantizer (FCVQ). The coding algorithm described displays good performance in the compression of test sequences at the rates of 8 and 16 kb/s; the signal-to-noise ratios obtained are good in both cases. The complexity of the coder implementation is comparable to that of conventional hybrid coders, while the decoder is much simpler in this proposal.

Analysis of object description methods in a video object tracking environment

A key issue in video object tracking is the representation of the objects and how effectively it discriminates between different objects. Several techniques have been proposed, but without a generally accepted method. While analysis and comparisons of these individual methods have been presented in the literature, their evaluation as part of a global solution has been overlooked. The appearance model for the objects is a component of a video object tracking framework, depending on previous processing stages and affecting those that succeed it. As a result, these interdependencies should be taken into account when analysing the performance of the object description techniques. We propose an integrated analysis of object descriptors and appearance models through their comparison in a common object tracking solution. The goal is to contribute to a better understanding of object description methods and their impact on the tracking process. Our contributions are threefold: propose a novel descriptor evaluation and characterisation paradigm; perform the first integrated analysis of state-of-the-art description methods in a scenario of people tracking; put forward some ideas for appearance models to use in this context. This work provides foundations for future tests and the proposed assessment approach contributes to the informed selection of techniques more adequately for a given tracking application context.

Filling the gap in quality assessment of video object tracking

Current evaluation methods either rely heavily on reference information manually annotated or, by completely avoiding human input, provide only a rough evaluation of the performance of video object tracking algorithms. The main objective of this paper is to present a novel approach to the problem of evaluating video object tracking algorithms. It is proposed the use different types of reference information and the combination of heterogeneous metrics for the purpose of approximating the ideal error. This will enable a significant decrease of the required reference information, thus bridging the gap between metrics with different requirements concerning this type of data. As a result, evaluation frameworks can aggregate the benefits from individual approaches while overcoming their weaknesses, providing a flexible and powerful tool to assess and characterize the behavior of the tracking algorithms.

CHAIR: automatic image registration based on correlation and Hough transform

Automatic image registration is a process related to several application fields: remote sensing, medicine and computer vision, among others. Particularly in the field of remote sensing, the ever-increasing number of available satellite images requires automatic image registration methods, capable of correctly aligning a new image. An automatic image registration method – CHAIR (correlation- and Hough transform-based method of automatic image registration) – is proposed, the key concept of which relies on the ‘correlation image’ produced in both the horizontal and vertical directions. In particular, the computation of the distance of an identified diagonal brighter strip in the correlation image (through the Hough transform) to an offset (the main diagonal) allows for the determination of translational shifts and consequently control points. The set of obtained control points allows for the correction of several types of distortions. The geometric correction quality achieved by CHAIR was objectively evaluated through measures recently proposed, which allow for a more complete assessment of the obtained results. The CHAIR performance was evaluated on both synthetic and real data, with different spatial resolutions and spectral contents. CHAIR has been shown to be able to correctly align two images with a subpixel accuracy, having a priori a ‘gold standard’ image covering a considerable part of the image to be registered, and has also been shown to work for images of different sensors and/or different spectral bands, situations where traditional correlation methods often yield low and smooth peaks on the correlation surface. It is also able to account for elevation differences and to some extent for rotation and scale effects. Furthermore, it has been shown to have potential for registering synthetic aperture radar (SAR) with optical images.

Object-Based Spatial Segmentation of Video Guided by Depth and Motion Information

Automatic spatial video segmentation is a problem without a general solution at the current state-of-the-art. Most of the difficulties arise from the process of capturing images, which remain a very limited sample of the scene they represent. The capture of additional information, in the form of depth data, is a step forward to address this problem. We start by investigating the use of depth data for better image segmentation; a novel segmentation framework is proposed, with depth being mainly used to guide a segmentation algorithm on the colour information. Then, we extend the method to also incorporate motion information in the segmentation process. The effectiveness and simplicity of the proposed method is documented with results on a selected set of images sequences. The achieved quality raises the expectation for a significant improvement on operations relying on spatial video segmentation as a pre-process.