Ekaterina Potapova

Learning what matters: combining probabilistic models of 2D and 3D saliency cues

In this paper we address the problem of obtaining meaningful saliency measures that tie in coherently with other methods and modalities within larger robotic systems. We learn probabilistic models of various saliency cues from labeled training data and fuse these into probability maps, which while appearing to be qualitatively similar to traditional saliency maps, represent actual probabilities of detecting salient features. We show that these maps are better suited to pick up task-relevant structures in robotic applications. Moreover, having true probabilities rather than arbitrarily scaled saliency measures allows for deeper, semantically meaningful integration with other parts of the overall system.

Attention-driven object detection and segmentation of cluttered table scenes using 2.5D symmetry

The task of searching and grasping objects in cluttered scenes, typical of robotic applications in domestic environments requires fast object detection and segmentation. Attentional mechanisms provide a means to detect and prioritize processing of objects of interest. In this work, we combine a saliency operator based on symmetry with a segmentation method based on clustering locally planar surface patches, both operating on 2.5D point clouds (RGB-D images) as input data to yield a novel approach to table-top scene segmentation. Evaluation on indoor table-top scenes containing man-made objects clustered in piles and dumped in a box show that our approach to selection of attention points significantly improves performance of state-of-the-art attention-based segmentation methods.

Saliency-based object discovery on RGB-D data with a late-fusion approach

We present a novel method based on saliency and segmentation to generate generic object candidates from RGB-D data. Our method uses saliency as a cue to roughly estimate the location and extent of the objects present in the scene. Salient regions are used to glue together the segments obtained from over-segmenting the scene by either color or depth segmentation algorithms, or by a combination of both. We suggest a late-fusion approach that first extracts segments from color and depth independently before fusing them to exploit that the data is complementary. Furthermore, we investigate several mechanisms for ranking the object candidates. We evaluate on one publicly available dataset and on one challenging sequence with a high degree of clutter. The results show that we are able to retrieve most objects in real-world indoor scenes and clearly outperform other state-of-the art methods.

Local 3d symmetry for visual saliency in 2.5d point clouds

Many models of visual attention have been proposed in the past, and proved to be very useful, e.g. in robotic applications. Recently it has been shown in the literature that not only single visual features, such as color, orientation, curvature, etc., attract attention, but complete objects do. Symmetry is a feature of many man-made and also natural objects and has thus been identified as a candidate for attentional operators. However, not many techniques exist to date that exploit symmetry-based saliency. So far these techniques work mainly on 2D data. Furthermore, methods, which work on 3D data, assume complete object models. This limits their use as bottom-up attentional operators working on RGBD images, which only provide partial views of objects. In this paper, we present a novel local symmetry-based operator that works on 3D data and does not assume any object model. The estimation of symmetry saliency maps is done on different scales to detect objects of various sizes. For evaluation a Winner-Take-All neural network is used to calculate attention points. We evaluate the proposed approach on two datasets and compare to state-of-the-art methods. Experimental results show that the proposed algorithm outperforms current state-of-the-art in terms of quality of fixation points.

Attention driven grasping for clearing a heap of objects

Generation of grasps for automated object manipulation in cluttered scenarios presents major challenges for various modules of the pipeline such as 2D/3D visual processing, 3D modeling, grasp hypothesis generation, grasp planning and path planning. In this paper, we present a solution framework for solving a complex instance of the problem - represented by a heap of unknown and unstructured objects in a bounded environment - in our case, a box; with the goal being removing all objects in the box using an attention driven object modeling approach to cognitive grasp planning. The focus of the algorithm delves on Grasping by Components (GBC), with a prioritization scheme derived from scene based attention and attention driven segmentation. In order to overcome the traditional challenge of segmentation performing poorly in cluttered scenes, we employ a novel active segmentation approach suited to our scenario. While the attention module helps prioritize objects in the heap and salient regions, the GBC scheme segments out parts and generates grasp hypotheses for each part. GBC is a very important component of any scalable and holistic grasping system since it abstracts point cloud object data with parametric shapes and no apriori knowledge (such as 3D models) is required. Earlier work in 3D model building (such as CAD based, simple geometries, bounding boxes, Superquadrics etc.) have depended on precise shape and pose recognition as well as exhaustive training to learn or exhaustive searching in grasp space to generate good grasp hypotheses. These methods are not scalable for real-time scenarios, complex shapes and unknown environments - key challenges in robotic grasping. In order to alleviate this concern, we present a novel parametric algorithm to estimate grasp points and approach vectors from the 3D parametric shape model, along with innovative schemes to optimize the computation of the parametric models as well as to refine the generated grasp hypotheses based on the scene information to aid path planning. We present evaluation of our complex grasping pipeline for cluttered heaps through a series of test sequences involving removal of objects from a box, along with evaluations for our attention mechanisms, active segmentation, 3D model fitting optimizations and quality of our grasp hypotheses.

Survey of recent advances in 3D visual attention for robotics

3D visual attention plays an important role in both human and robotics perception that yet has to be explored in full detail. However, the majority of computer vision and robotics methods are concerned only with 2D visual attention. This survey presents findings and approaches that cover 3D visual attention in both human and robot vision, summarizing the last 30 years of research and also looking beyond computational methods. First, we present work in such fields as biological vision and neurophysiology, studying 3D attention in human observers. This provides a view of the role attention plays at the system level for biological vision. Then, we cover computer and robot vision approaches that take 3D visual attention into account. We compare approaches with respect to different categories, such as feature-based, data-based, or depth-based visual attention, and draw conclusions on what advances will help robotics to cope better with complex real-world settings and tasks.

Incremental attention-driven object segmentation

Segmentation of highly cluttered indoor scenes is a challenging task and should be solved in real time to be efficiently used in such applications as robotics, for example. Traditional segmentation methods are often overwhelmed by the complexity of the scene and require significant processing time. To tackle this problem we propose to use incremental attention-driven segmentation, where attention mechanisms are used to prioritize parts of the scene to be handled first. Our method outputs object hypotheses composed of parametric surface models. We evaluate our approach on two publicly available datasets of cluttered indoor scenes. We show that the proposed method outperforms existing methods of attention-driven segmentation in terms of segmentation quality and computational performance.

Workshop on attention models in robotics: visual systems for better HRI

Attention is a concept of human perception that enables human subjects to select the potentially relevant parts out of the huge amount of sensory data and that enables interactions with other human subjects by sharing attention with each other. These abilities are also of large interest for autonomous robots, therefore, interest in modeling concepts of human attention computationally has increased strongly in the robotics community during the last decade. Especially in human-robot interaction, the ability to detect what a human partner is attending to and to act in a similar way to enable intuitive communication, are important skills for a robotic system.Still, there exists a gap in knowledge transfer between researchers in human attention and robotic researchers with their specific, often task-related, problems. Both communities can mutually benefit from each other by sharing ideas. In the workshop, researchers in visual and multi-modal attention can profit from the rapidly growing field of robotics, which offers new and challenging research questions with very concrete applicability to challenging problems. Robotic researchers can learn how to integrate attention to support natural and real-time HRI.Categories and Subject DescriptorsI.2.m [Artificial Intelligence]: Miscellaneous

Situiertes Sehen für bessere Erkennung von Objekten und Objektklassen

SummaryA main task for domestic robots is to recognize object categories. Image-based approaches learn from large data bases but have no access to contextual knowledge such as available to a robot navigating in the rooms at home. Consequently, we set out to exploit the knowledge available to the robot to constrain the task of object classification. Based on the estimation of free ground space in front of the robot, which is essential for the save navigation in a home setting, we show that we can greatly simplify self-localization, the detection of support surfaces, and the classification of objects. We further show that object classes can be efficiently acquired from 3D models of the Web if learned from automatically generated view data. We modelled 200 object classes (available from www.3d-net.org) and provide sample data of scenes for testing. Using the situated approach we can detect, e.g., chairs with 93 per cent detection rate.ZusammenfassungEine Hauptaufgabe für Roboter ist es, Objekte und Objektklassen zu erkennen, um diese zu finden und handzuhaben. Bild-basierte Ansätze lernen aus großen Datenbanken, haben aber keinen Zugriff auf Kontextwissen zur Verfügung, zum Beispiel, wie Roboter in Zimmern navigieren. Wir schlagen daher den Ansatz des situierten Sehens vor, um kontextuelles Wissen über die Aufgabe und die Anwendung zur Verbesserung der Objekterkennung zu verwenden. Basierend auf der Bestimmung des freien Bodens vor dem Roboter, der für die sichere Navigation notwendig ist, zeigen wir, dass dadurch die Lokalisierung, das Erkennung von Flächen und die Kategorisierung von Objekten vereinfacht werden. Wir zeigen ferner, dass Objektklassen effizient aus 3D-Web-Daten gelernt werden können, wenn das Lernen virtuelle 2,5D-Ansichten verwendet, um die Sicht der Sensoren des Roboters auf die reale Welt anzunehmen. Mit diesem Ansatz wurden 200 Objektklassen (zu finden unter www.3d-net.org) modelliert und in Szenen erkannt, z. B. Stühle mit einer Erkennungsrate von 93 Prozent.