Andrea Carbone

Bottom-Up Gaze Shifts and Fixations Learning by Imitation

The ability to follow the gaze of conspecifics is a critical component in the development of social behaviors, and many efforts have been directed to studying the earliest age at which it begins to develop in infants. Developmental and neurophysiological studies suggest that imitative learning takes place once gaze-following abilities are fully established and joint attention can support the shared behavior required by imitation. Accordingly, gaze-following acquisition should be precursory to most machine learning tasks, and imitation learning can be seen as the earliest modality for acquiring meaningful gaze shifts and for understanding the structural substrate of fixations. Indeed, if some early attentional process, based on a suitable combination of gaze shifts and fixations, could be learned by the robot, then several demonstration learning tasks would be dramatically simplified. In this paper, we describe a methodology for learning gaze shifts based on imitation of gaze following with a gaze machine, which we purposefully introduced to make the robot gaze imitation conspicuous. The machine allows the robot to share and imitate gaze shifts and fixations of a caregiver through a mutual vergence. This process is then suitably generalized by learning both the scene salient features toward which the gaze is directed and the way saccadic programming is attained. Salient features are modeled by a family of Gaussian mixtures. These together with learned transitions are generalized via hidden Markov models to account for humanlike gaze shifts allowing to discriminate salient locations

Motion Saliency Maps from Spatiotemporal Filtering

For artificial systems acting and perceiving in a dynamic world a core ability is to focus on aspects of the environment that can be crucial for the task at hand. Perception in autonomous systems needs to be filtered by a biologically inspired selective ability, therefore attention in dynamic settings is becoming a key research issue. In this paper we present a model for motion salience map computation based on spatiotemporal filtering. We extract a measure of coherent motion energy and select by the center-surround mechanism relevant zones that accumulate most energy and therefore contrast with surroundings in a given time slot. The method was tested on synthetic and real video sequences, supporting biological plausibility.

Augmenting situation awareness via model-based control in rescue robots

In this work we describe a model-based approach to the executive control of a rescue rover. We show how this control architecture naturally supports human-robot interaction in the diverse activities needed in rescue and search. We illustrate the approach by considering human-robot interaction in the domain of the RoboCup rescue competition. We discuss the implementation and tests done both during RoboCup contests and in the laboratory, to show performances according to different working modalities such as fully operated, supervised, fully autonomous.

Robot task-driven attention

Visual attention is a crucial skill in human beings in that it allows optimal deployment of visual processing and memory resources. It turns out to be even more useful in search tasks, since to select salient zones we use top-down priors, depending on the observed scene, along with bottom-up criteria. In this paper we show how we constructed a robotic model of attention, inspired by studies on human attention and gaze shifting. Our model relies on a measure of salience related to the particular type of environment and to the given task. This measure is hierarchically structured and consists of both top-down components, learned from the tutor, and bottom-up components as perceived in the scene by the robot. Hence with such a general model the robot can perform its own scan-path inside a similar environment and report on its findings.

Situation aware rescue robots

We present a model-based approach to the execution and control of an autonomous system, based on flexible behaviours, and supporting a novel view of human-robot interaction. The well-known RoboCup rescue competition is also discussed and presented as a special case for testing human-robot interaction architectures. The activities of the system, according to the components performing them, are modeled as flexible behaviours and are executed according to different modalities, such as fully operated, supervised, fully autonomous. We finally discuss the implementation and tests done both during the contests and in the laboratory, to show performances according to the different modalities.

Computationally effective stereovision SLAM

In this paper a visual Simultaneous Localization and Mapping (SLAM) algorithm suitable for indoor area measurement applications is proposed. The algorithm is focused on computational effectiveness. The only sensor used is a stereo camera placed onboard a moving robot. The algorithm processes the acquired images calculating the depth of the scenery, detecting occupied areas and progressively building a map of the environment. The stereo vision-based SLAM algorithm embodies a custom-tailored stereo correspondence algorithm, the robust scale and rotation invariant feature detection and matching Speeded Up Robust Features (SURF) method, a computationally effective v-disparity image calculation scheme, a novel map-merging module, as well as a sophisticated Cellular Automata (CA)-based enhancement stage. The proposed algorithm is suitable for autonomously mapping and measuring indoor areas using robots. The algorithm is presented and experimental results for self-captured image sets are provided and analyzed.

Autonomous Attentive Exploration in Search and Rescue Scenarios

In task-oriented exploration a robot has to direct its sight and delving towards the most promising regions of the environment, according to the task, in order to optimize its search. If the goal is dynamically set on the basis of what it is perceived, attention plays a crucial role, as it allows to combine fast glancing with accurate analysis, enabling the robot to quickly jump to conclusion by selecting the interesting spots in the environment requiring a further analysis. We present a new approach to attentive exploration designed for an autonomous rover working in rescue scenarios. The visual-attention process combined with the simultaneous localization and mapping one guides the robot search through an incremental generation of a view-pointsaliency map obtained according to transportation processes. Interesting features emerging from pre-attentive pop-outs are projected on the current metric map and, according the preference engendered, diffuse streams of particles warming up those map areas they pass through, in so generating hot regions that result in optimal vantage points for the robot to observe the salient spots glanced at while searching. We show the effectiveness of the approach by providing experimental results and comparisons.

Spatial discrimination in task-driven attention

Visual attention is becoming an increasingly imperative capability to endow computer vision systems and autonomous agents with. Starting from a biological inspired model of attention, we present an experiment aimed to study selective attention in 3D space. Depth has been proved to be an important feature affecting the way attention is deployed when observing a scene. We studied preferential scanning paths and fixation zones in a task-driven wandering of the tutor gaze over a scene where multiple targets had been disposed on different depth planes. We supposed that selective attention would aggregate targets in cliques that maximize utility, minimizing meanwhile visual effort produced when passing from closer planes to further planes or between different cluttered locations. By means of a purposely designed machine, we stored visual and motor data of the tutors head; we clustered different scanning paths of the gaze shifts according to velocity and space criteria to determine a preference model of attentional shifts and fixations. We propose subsequently a utility model that can formalize acquired information and establish a vision-based attentional framework for robots. We show that an interpretation of task-driven gaze orienting based on the presented preference criteria correctly accounts for the studied behaviours, as further reported in the literature

Gaze motion clustering in scan-path estimation.

Visual attention is considered nowadays a paramount ability both in Cognitive Sciences and in Cognitive Vision to bridge the gap between perception and higher level reasoning functions, such as scene interpretation and decision making. Bottom-up gaze shifting is the main mechanism used by humans when exploring a scene without a specific task. In this paper we investigated which criteria allow for the generation of plausible fixation clusters by analysing experimental data of human subjects. We suggest that fixations should be grouped in cliques whose saliency can be assessed through an innovation factor encompassing bottom-up cues, proximity, direction and memory components.

Humeral head inferior subluxation in proximal humerus fractures

PurposeThe objectives of this study are to propose a reliable radiologic method for detecting static inferior subluxation of humeral head, to calculate the relative intra- and inter-observer reliability, and to evaluate its presence pre- and post-surgery.MethodsThis is a retrospective observational study of patients surgically treated for a fracture of the proximal humerus. Fractures were classified using Codman-Lego criteria (radiographic, CT images), osteoporosis was assessed. To identify inferior subluxation, an original method is proposed. This measurement was done pre-operatively, at three and 12 month post-operatively. Clinical evaluation was recorded at final follow-up using Constant Score.ResultsOne hundred fifty fractures surgically treated were studied. Intra- and inter-observer reliabilities were excellent and high, respectively. In pre-operative x-rays, a significant inferior subluxation was noted in 17/150 cases (11.3%), with significant correlation with fracture pattern (p=0.045), female sex (p=0.038), age older than 70 (p=0.003), obesity (BMI>30, p=0.03), and local osteoporosis (p=0.002). At three month of follow-up, 22 cases (14.6%) had inferior subluxation, with significant correlation with female sex (p=0.04), age older than 70 (p=0.002), obesity (p=0.02), pin or screw articular surface perforation (p<0.001). At 12 month of follow-up, seven cases showed persistent inferior subluxation, with significant correlation with age older than 70 (p=0.032), obesity (p=0.041), screw joint perforation and lower Constant Score (p<0.001).DiscussionInferior subluxation was mostly found in osteoporotic fractures of the elderly, obese, and of female sex both pre- and post-operatively. The intra- and inter-observer reliabilities of proposed radiographic measurement were high and excellent, respectively.ConclusionsIn the early postoperative months, we found a high correlation between inferior subluxation and articular surface perforation; when persisting at later follow-ups, we might speculate that it could represent an early phase of avascular necrosis of the humeral head.Level of evidenceLevel III, observational study.