Georgios Pavlakos

Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose

This paper addresses the challenge of 3D human pose estimation from a single color image. Despite the general success of the end-to-end learning paradigm, top performing approaches employ a two-step solution consisting of a Convolutional Network (ConvNet) for 2D joint localization and a subsequent optimization step to recover 3D pose. In this paper, we identify the representation of 3D pose as a critical issue with current ConvNet approaches and make two important contributions towards validating the value of end-to-end learning for this task. First, we propose a fine discretization of the 3D space around the subject and train a ConvNet to predict per voxel likelihoods for each joint. This creates a natural representation for 3D pose and greatly improves performance over the direct regression of joint coordinates. Second, to further improve upon initial estimates, we employ a coarse-to-fine prediction scheme. This step addresses the large dimensionality increase and enables iterative refinement and repeated processing of the image features. The proposed approach outperforms all state-of-the-art methods on standard benchmarks achieving a relative error reduction greater than 30% on average. Additionally, we investigate using our volumetric representation in a related architecture which is suboptimal compared to our end-to-end approach, but is of practical interest, since it enables training when no image with corresponding 3D groundtruth is available, and allows us to present compelling results for in-the-wild images.

6-DoF object pose from semantic keypoints

This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior work, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Empirically, we show that the proposed approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios with a cluttered background. For class-based object pose estimation, state-of-the-art accuracy is shown on the large-scale PASCAL3D+ dataset.

Advances in Intelligent Mobility Assistance Robot Integrating Multimodal Sensory Processing

Mobility disabilities are prevalent in our ageing society and impede activities important for the independent living of elderly people and their quality of life. The goal of this work is to support human mobility and thus enforce fitness and vitality by developing intelligent robotic platforms designed to provide user-centred and natural support for ambulating in indoor environments. We envision the design of cognitive mobile robotic systems that can monitor and understand specific forms of human activity, in order to deduce what the human needs are, in terms of mobility. The goal is to provide user and context adaptive active support and ambulation assistance to elderly users, and generally to individuals with specific forms of moderate to mild walking impairment. To achieve such targets, a reliable multimodal action recognition system needs to be developed, that can monitor, analyse and predict the user actions with a high level of accuracy and detail. Different modalities need to be combined into an integrated action recognition system. This paper reports current advances regarding the development and implementation of the first walking assistance robot prototype, which consists of a sensorized and actuated rollator platform. The main thrust of our approach is based on the enhancement of computer vision techniques with modalities that are broadly used in robotics, such as range images and haptic data, as well as on the integration of machine learning and pattern recognition approaches regarding specific verbal and non-verbal gestural commands in the envisaged physical and non-physical human-robot interaction context.

Harvesting Multiple Views for Marker-Less 3D Human Pose Annotations

Recent advances with Convolutional Networks (ConvNets) have shifted the bottleneck for many computer vision tasks to annotated data collection. In this paper, we present a geometry-driven approach to automatically collect annotations for human pose prediction tasks. Starting from a generic ConvNet for 2D human pose, and assuming a multi-view setup, we describe an automatic way to collect accurate 3D human pose annotations. We capitalize on constraints offered by the 3D geometry of the camera setup and the 3D structure of the human body to probabilistically combine per view 2D ConvNet predictions into a globally optimal 3D pose. This 3D pose is used as the basis for harvesting annotations. The benefit of the annotations produced automatically with our approach is demonstrated in two challenging settings: (i) fine-tuning a generic ConvNet-based 2D pose predictor to capture the discriminative aspects of a subjects appearance (i.e.,personalization), and (ii) training a ConvNet from scratch for single view 3D human pose prediction without leveraging 3D pose groundtruth. The proposed multi-view pose estimator achieves state-of-the-art results on standard benchmarks, demonstrating the effectiveness of our method in exploiting the available multi-view information.

Towards an intelligent robotic walker for assisted living using multimodal sensorial data

We aim at developing an intelligent robotic platform that provides cognitive assistance and natural support in indoor environments to the elderly society and to individuals with moderate to mild walking impairment. Towards this end, we process data from audiovisual sensors and laser range scanners, acquired in experiments with patients in real life scenarios. We present the main concepts of an automatic system for user intent and action recognition that will integrate multiple modalities. We demonstrate promising preliminary results, firstly on action recognition based on the visual modality, i.e. color and depth cues, and secondly on the detection of gait cycle patterns that exploit the laser range data. For action recognition we are based on local interest points, 3D Gabor filters and dominant energy analysis, feeding a support vector machine. Then the recognized actions can trigger the gait cycle detection that detect walking patterns by exploiting the laser range data, modeled by hidden Markov models. In this way, we shall acquire the overall patients state and the robot shall autonomously reason on how to provide support.

Reconstruction of 3D Pose for Surfaces of Revolution from Range Data

Axial symmetry is a common property of everyday objects. Bottles, cups, cans and bowls, all usually fall in that category and can be modeled by surfaces of revolution (SOR). In this paper, we address the problem of estimating the parameters of an SOR (axis and generatrix) from range data. Although SOR reconstruction from RGB images is well studied, previous works using depth measurements are limited. We propose a formulation similar to the 3D registration problem and our solution is based on an alternating procedure that recovers the complete surface geometry, i.e. The axis and the profile curve of the SOR. We evaluate our method both quantitatively and qualitatively using four different datasets that provide depth images from a large variety of axially symmetric objects.