Koen Buys

An adaptable system for RGB-D based human body detection and pose estimation

HighlightsDoes not require pre-processing by background subtraction and no initialization poses.Online learned appearance model combining color with depth-based labeling.Works in clutter and with body part occlusions because of underlying kinematic model.RDF training, data generation and cluster-based learning, that enables retraining. Human body detection and pose estimation is useful for a wide variety of applications and environments. Therefore a human body detection and pose estimation system must be adaptable and customizable. This paper presents such a system that extracts skeletons from RGB-D sensor data. The system adapts on-line to difficult unstructured scenes taken from a moving camera (since it does not require background subtraction) and benefits from using both color and depth data. It is customizable by virtue of requiring less training data, having a clearly described training method, and a customizable human kinematic model. Results show successful application to data from a moving camera in cluttered indoor environments. This system is open-source, encouraging reuse, comparison, and future research.

Multi RGB-D camera setup for generating large 3D point clouds

The advent of inexpensive RGB-D cameras brings new opportunities to capture a 3D environment. This paper presents a method to create a modular setup for generating a large 3D point cloud, with attention to the study of interference, the influence of a USB extension cable, and the calibration procedure. The study of interference includes the influence of the distance between the cameras, the orientation of the cameras, and the illumination. Furthermore, this paper proposes a number of evaluation metrics for similar setups.

Parametric Modeling of the Human Body Using Kinect Measurements

Personalized digital human modeling is useful for a wide variety of applications. An obvious interest comes from the entertainment industry, where movies and video games explore the possibilities of this technology. In biomechanics, human modeling assists in the design of person-specific solutions to improve the human well-being and ergonomics. Other applications exist in virtual dressing rooms, human-robot interactions in robotics, etc. Existing technology, although performing well, has the disadvantage of being expensive, immobile, not fully customizable and possibly requiring external body markers. To tackle these issues, this paper presents a modeling technique using the open source software MakeHuman, based on body measurements obtained with Microsoft’s Kinect. The current solution is able to retrieve these measurements when the person is standing in a calibrated scene, this means when the person’s position is known a priori. In order to retrieve the measurement data as a point cloud, and to process this point cloud, the PCL (Point Cloud Library) software is used, leading to a fully open source implementation. With these tools, solutions for person segmentation, measuring and personalized modeling are proposed. It appears that the current Kinect technology on itself is not very accurate for measuring body sizes. However, this work shows that the Kinect information combined with the MakeHuman modeling tool is valuable. The final model incorporates measures like body height, arm span, hip, waist and chest width, completed with information such as age, gender and weight. Evaluation of the resulting human model shows moderate to good results in modeling body height, hip and waist width, whereas chest width modeling is rather poor due to difficulties in chest width extraction from Kinect images..

Haptic coupling with augmented feedback between two KUKA Light-Weight Robots and the PR2 robot arms

This paper discusses the theoretical background and practical implementation of a large-scale, low-performance haptic remote control setup. The experimental system consists of a pair of KUKA Light Weight Robots (LWR) coupled to a Willow Garage Personal Robot (PR2) via two different robotic frameworks. The haptic “performance” is, of course, not comparable to dedicated haptic applications, but has its use as a test-bed for interaction between “legacy” service robot systems, that have not been especially designed for mutual haptic interaction. We discuss some major application problems, and the future work needed for nonuniform robot coupling. Beside haptic coupling, we provide the human operator with visual feedback. To this end, the head movements of the human operator are coupled to the head movement of the PR2 and the images of the eye cameras are displayed to the human operator using a wearable display. The presented teleoperation application is furthermore an example of the integration of two component-based robotic frameworks namely OROCOS (Open Robot Control Software)and ROS (Robot Operating System) Experimental results regarding the haptic coupling are presented using an “artistic” painting task for qualitative results, and a hard contact at the slave side for quantitative results.