Germán Martín García

Traditional saliency reloaded: A good old model in new shape

In this paper, we show that the seminal, biologically-inspired saliency model by Itti et al. [21] is still competitive with current state-of-the-art methods for salient object segmentation if some important adaptions are made. We show which changes are necessary to achieve high performance, with special emphasis on the scale-space: we introduce a twin pyramid for computing Difference-of-Gaussians, which enables a flexible center-surround ratio. The resulting system, called VOCUS2, is elegant and coherent in structure, fast, and computes saliency at the pixel level. It is not only suitable for images with few objects, but also for complex scenes as captured by mobile devices. Furthermore, we integrate the saliency system into an object proposal generation framework to obtain segment-based saliency maps and boost the results for salient object segmentation. We show that our system achieves state-of-the-art performance on a large collection of benchmark data.

A Cognitive Approach for Object Discovery

Object discovery is the task of detecting unknown objects in images. The task is of large interest in many fields of machine vision, ranging from the automatic analysis of web images to interpreting data of a mobile robot or a driver assistant system. Here, we present a new approach for object discovery, based on findings of the human visual system. Proto-objects are detected with a segmentation module, generating perceptually coherent image regions. In parallel, a saliency system detects regions of interest in images and serves to select segments, depending on their saliency. We obtain very good results on a database of salient objects and on real-world office scenes.

Adaptive Multi-cue 3D Tracking of Arbitrary Objects

We present a general method for RGB-D data that is able to track arbitrary objects in real-time in challenging real-world scenarios. The method is based on the Condensation algorithm. The observation model consists of a target/background classifier that is boosted from a pool of grayscale, color, and depth features. The training set of the observation model is updated with new examples from tracking and the classifier is re-trained to cope with the new appearances of the target. A mechanism maintains a small set of specialized candidate features in the pool, thus decreasing the computational time, while keeping the performance stable. Depth measurements are integrated into the prediction of the 3D state of the particles. We evaluate our approach with a new benchmark for RGB-D tracking algorithms; the results prove our method to be robust under real-world settings, being able to keep track of the targets over 96% of the time.

Saliency-based object discovery on RGB-D data with a late-fusion approach

We present a novel method based on saliency and segmentation to generate generic object candidates from RGB-D data. Our method uses saliency as a cue to roughly estimate the location and extent of the objects present in the scene. Salient regions are used to glue together the segments obtained from over-segmenting the scene by either color or depth segmentation algorithms, or by a combination of both. We suggest a late-fusion approach that first extracts segments from color and depth independently before fusing them to exploit that the data is complementary. Furthermore, we investigate several mechanisms for ranking the object candidates. We evaluate on one publicly available dataset and on one challenging sequence with a high degree of clutter. The results show that we are able to retrieve most objects in real-world indoor scenes and clearly outperform other state-of-the art methods.

Sequence-level object candidates based on saliency for generic object recognition on mobile systems

In this paper, we propose a novel approach for generating generic object candidates for object discovery and recognition in continuous monocular video. Such candidates have recently become a popular alternative to exhaustive window-based search as basis for classification. Contrary to previous approaches, we address the candidate generation problem at the level of entire video sequences instead of at the single image level. We propose a processing pipeline that starts from individual region candidates and tracks them over time. This enables us to group candidates for similar objects and to automatically filter out inconsistent regions. For generating the per-frame candidates, we introduce a novel multi-scale saliency approach that achieves a higher per-frame recall with fewer candidates than current state-of-the-art methods. Taken together, those two components result in a significant reduction of the number of object candidates compared to frame level methods, while keeping a consistently high recall.

Attention-Based Detection of Unknown Objects in a Situated Vision Framework

We present an attention-based approach for the detection of unknown objects in a 3D environment. The ability to address individual objects in the environment without having previous knowledge about their properties or their identity is one important requirement of the Situated Vision theory. Based on saliency maps, our attention system determines the regions where objects are likely to be found; these are the proto-objects whose extent is refined by a 2D segmentation step. At the same time a 3D scene model is built from measurements of a depth camera. The detected objects are projected into the 3D scene, resulting in 3D object models which are incrementally updated. We show the validity of our approach in an RGB-D sequence recorded in an office environment.

Semantic segmentation priors for object discovery

Reliable object discovery in realistic indoor scenes is a necessity for many computer vision and service robot applications. In these scenes, semantic segmentation methods have made huge advances in recent years. Such methods can provide useful prior information for object discovery by removing false positives and by delineating object boundaries. We propose a novel method that combines bottom-up object discovery and semantic priors for producing generic object candidates in RGB-D images. We use a deep learning method for semantic segmentation to classify colour and depth superpixels into meaningful categories. Separately for each category, we use saliency to estimate the location and scale of objects, and superpixels to find their precise boundaries. Finally, object candidates of all categories are combined and ranked. We evaluate our approach on the NYU Depth V2 dataset and show that we outperform other state-of-the-art object discovery methods in terms of recall.

A computational framework for attentional object discovery in RGB-D videos

AbstractWe present a computational framework for attention-guided visual scene exploration in sequences of RGB-D data. For this, we propose a visual object candidate generation method to produce object hypotheses about the objects in the scene. An attention system is used to prioritise the processing of visual information by (1) localising candidate objects, and (2) integrating an inhibition of return (IOR) mechanism grounded in spatial coordinates. This spatial IOR mechanism naturally copes with camera motions and inhibits objects that have already been the target of attention. Our approach provides object candidates which can be processed by higher cognitive modules such as object recognition. Since objects are basic elements for many higher level tasks, our architecture can be used as a first layer in any cognitive system that aims at interpreting a stream of images. We show in the evaluation how our framework finds most of the objects in challenging real-world scenes.

Attentional Scene-Exploration and Object Discovery in Image and RGB-D Data

In this paper, we summarize our project work of the last two years, where we addressed the tasks of visually exploring a scene with visual attention mechanisms based on saliency computation, and of locating unknown objects in the environment. The latter is also called object discovery and consists in finding candidate objects without previous knowledge about the objects themselves or the scene. We follow an approach motivated from human perception and combine saliency and segmentation to generate object candidates. We show results on 2D images as well as on 3D sequences obtained from an RGB-D camera.