Dimitrios Kanoulas

WALK‐MAN: A High‐Performance Humanoid Platform for Realistic Environments

In this work, we present WALK-MAN, a humanoid platform that has been developed to operate in realistic unstructured environment, and demonstrate new skills including powerful manipulation, robust balanced locomotion, high-strength capabilities, and physical sturdiness. To enable these capabilities, WALK-MAN design and actuation are based on the most recent advancements of series elastic actuator drives with unique performance features that differentiate the robot from previous state-of-the-art compliant actuated robots. Physical interaction performance is benefited by both active and passive adaptation, thanks to WALK-MAN actuation that combines customized high-performance modules with tuned torque/velocity curves and transmission elasticity for high-speed adaptation response and motion reactions to disturbances. WALK-MAN design also includes innovative design optimization features that consider the selection of kinematic structure and the placement of the actuators with the body structure to maximize the robot performance. Physical robustness is ensured with the integration of elastic transmission, proprioceptive sensing, and control. The WALK-MAN hardware was designed and built in 11 months, and the prototype of the robot was ready four months before DARPA Robotics Challenge (DRC) Finals. The motion generation of WALK-MAN is based on the unified motion-generation framework of whole-body locomotion and manipulation (termed loco-manipulation). WALK-MAN is able to execute simple loco-manipulation behaviors synthesized by combining different primitives defining the behavior of the center of gravity, the motion of the hands, legs, and head, the body attitude and posture, and the constrained body parts such as joint limits and contacts. The motion-generation framework including the specific motion modules and software architecture is discussed in detail. A rich perception system allows the robot to perceive and generate 3D representations of the environment as well as detect contacts and sense physical interaction force and moments. The operator station that pilots use to control the robot provides a rich pilot interface with different control modes and a number of teleoperated or semiautonomous command features. The capability of the robot and the performance of the individual motion control and perception modules were validated during the DRC in which the robot was able to demonstrate exceptional physical resilience and execute some of the tasks during the competition.

Detecting object affordances with Convolutional Neural Networks

We present a novel and real-time method to detect object affordances from RGB-D images. Our method trains a deep Convolutional Neural Network (CNN) to learn deep features from the input data in an end-to-end manner. The CNN has an encoder-decoder architecture in order to obtain smooth label predictions. The input data are represented as multiple modalities to let the network learn the features more effectively. Our method sets a new benchmark on detecting object affordances, improving the accuracy by 20% in comparison with the state-of-the-art methods that use hand-designed geometric features. Furthermore, we apply our detection method on a full-size humanoid robot (WALK-MAN) to demonstrate that the robot is able to perform grasps after efficiently detecting the object affordances.

Preparatory object reorientation for task-oriented grasping

This paper describes a new task-oriented grasping method to reorient a rigid object to its nominal pose, which is defined as the configuration that it needs to be grasped from, in order to successfully execute a particular manipulation task. Our method combines two key insights: (1) a visual 6 Degree-of-Freedom (DoF) pose estimation technique based on 2D-3D point correspondences is used to estimate the object pose in real-time and (2) the rigid transformation from the current to the nominal pose is computed online and the object is reoriented over a sequence of steps. The outcome of this work is a novel method that can be effectively used in the preparatory phase of a manipulation task, to permit a robot to start from arbitrary object placements and configure the manipulated objects to the nominal pose, as required for the execution of a subsequent task. We experimentally demonstrate the effectiveness of our approach on a full-size humanoid robot (WALK-MAN) using different objects with various pose settings under real-time constraints.

Visual Grasp Affordance Localization in Point Clouds Using Curved Contact Patches

Detecting affordances on objects is one of the main open problems in robotic manipulation. This paper presents a new method to represent and localize grasp affordances as bounded curved contact patches (paraboloids) of the size of the robotic hand. In particular, given a three-dimensional (3D) point cloud from a range sensor, a set of potential grasps is localized on a detected object by a fast contact patch fitting and validation process. For the object detection, three standard methods from the literature are used and compared. The potential grasps on the object are then refined to a single affordance using their shape (size and curvature) and pose (reachability and minimum torque effort) properties, with respect to the robot and the manipulation task. We apply the proposed method to a circular valve turning task, verifying the ability to accurately and rapidly localize grasp affordances, under significant uncertainty in the environment. We experimentally validate the method with the humanoid robot COMAN on 10 circular control valves fixed on a wall, from five different viewpoints and robot poses for each valve. We compare the reliability of the introduced local grasp affordances method to the baseline that relies only on object detection, illustrating the superiority of ours for the valve turning task.

Uncertainty analysis for curved surface contact patches

We present a Gaussian uncertainty analysis of bounded curved patches that fit to local rough surfaces and are suitable for representing foothold or handhold contacts between an articulated robot and the environment. The input is a set of 3D point samples with 3×3 covariance matrices that express their Gaussian uncertainty. We first introduce uncertainty propagation of geometrical patch parameters during fitting on range samples. The output for each patch includes a covariance matrix in its parametric space. We also introduce a set of distance metrics to validate the magnitude of the propagated uncertainty and we run a set of tests on various range data. The importance of this paper lies in the uncertainty analysis for curved contact patches that can be further applied during locomotion or manipulation.

An active compliant impact protection system for humanoids: Application to WALK-MAN hands

This paper reports on the development of a new pneumatically actuated impact protection system which can be applied to protect humanoid robots during high impact physical interactions. The proposed device is based on a soft inflating vessel which is integrated and validated on the hands of a humanoid robot WALK-MAN. The system incorporates an active pressure control unit with on-off solenoid valves that permit the regulation of the air pressure of the protection chamber. The impact protection system is smaller and lighter than a rubber-based passive protection previously mounted on the hands, while it offers better impact reduction performance via fast and accurate pressure control. The effectiveness of the system is verified by actual physical interaction experiments with WALK-MAN while the robot is falling against an inclined surface, making contact with its hands to support its body and prevent falling and damage.

Terrain classification and locomotion parameters adaptation for humanoid robots using force/torque sensing

This paper describes a terrain classification method based on the readings from the force/torque sensors mounted on the ankles of a humanoid robot. The experimental results on five different terrain types, showed very high precision and recall identification rates, i.e. 95%, that are surpassing the state-of-the-art ones for quadrupeds and hexapods. Based on the acquired data during a set of walking experiments, we evaluated the stability of locomotion on all of the terrains. We also present a method to find an optimal step size, which optimises both the energy consumption and the stability of locomotion, given the identified terrain type. For the experimental data collection we used the full-size humanoid robot WALK-MAN walking on five different types of terrain.

An affordance-based pilot interface for high-level control of humanoid robots in supervised autonomy

In this work we present the concept of a pilot interface to control a humanoid robot on an abstract level in unknown environments. The environment is perceived with a stereo camera system and then simplified into a set of environmental primitives. Based on these primitives the interface proposes affordances to the pilot. Affordances are represented as certainty functions over the space of end-effector poses. The pilot operates the robot by selecting among proposed affordances and related action primitives, i.e. Object-Action Complexes (OACs). Before initiating execution, the pilot can review and revise the parameterization of the scheduled action primitive in a 3D reconstruction of the environment. The pilot interface proposed in this work has been implemented and evaluated on the humanoid robot WALK-MAN. With this work we also demonstrate the transferability of the perceptual concept, as our previous experiments have been performed using the humanoid robot ARMAR-III.

WALK-MAN Humanoid Platform

In this chapter we present WALK-MAN, a humanoid platform that has been developed to operate in realistic unstructured environments and demonstrate new skills including powerful manipulation, robust balanced locomotion, high strength capabilities and physical sturdiness. To enable these capabilities, WALK-MAN design and actuation are based on the most recent advancements of Series Elastic Actuation (SEA) drives with unique performance features that differentiate the robot from previous state-of-the-art compliant actuated robots. Physical interaction performance benefits from both active and passive adaptation thanks to WALK-MAN actuation, which combines customized high performance modules with tuned torque/velocity curves and transmission elasticity for high speed adaptation response and motion reactions to disturbances. The WALK-MAN design also includes innovative design optimization features that consider the selection of kinematic structure and the placement of the actuators with respect to the body structure to maximize the robot performance. Physical robustness is ensured with the integration of elastic transmission, proprioceptive sensing and control. WALK-MAN hardware was designed and built in 11 months, and the prototype of the robot was ready 4 months before the DARPA Robotics Challenge (DRC) Finals. The motion generation of WALK-MAN is based on the unified motion generation framework of whole-body locomotion and manipulation (termed loco-manipulation). WALK-MAN is able to execute simple loco-manipulation behaviours synthesized by combining different primitives defining the behaviour of the center of gravity, of the hands, legs and head, the body attitude and posture, and the constrained body parts such as joint limits and contacts. The motion generation framework including the specific motion modules and software architecture are discussed in detail. A rich perception system allows the robot to perceive and generate 3D representations of the environment as well as detect contacts and sense physical interaction force and moments. The operator station that pilots use to control the robot provides a rich pilot interface with different control modes and a number of tele-operated or semi-autonomous command features. The capability of the robot and the performance of the individual motion control and perception modules were validated during the DARPA Robotics Challenge in which the robot was able to demonstrate exceptional physical resilience and execute some of the tasks during the competition.

Humanoids at Work: The WALK-MAN Robot in a Postearthquake Scenario

Nowadays human intervention is the only effective course of action after a natural or artificial disaster. This is true both for the relief operations where search–and–rescue of survivors is the priority, and for subsequent activities such as the ones devoted to building assessment. In these contexts the use of robotic systems would be beneficial to drastically reduce operators’ risk exposure. The readiness level of the robots still prevents their effective exploitation in relief operations, that are highly critical and characterized by severe time constraints. On the contrary current robotic technologies can be profitably applied in procedures like building assessment after an earthquake. To date, these operations are carried out by engineers and architects who inspect numerous buildings over a large territory, with a high cost in terms of time and assets, and with a high risk due to aftershocks. The main idea is to have the robot acting as an alter-ego of the human operator, who, thanks to a virtual reality device and a body tracking system based on inertial sensors, teleoperates the robot. The goal of this paper is to exploit the perception and manipulation capabilities of the WALK-MAN robot for building assessment in areas affected by earthquakes. The presented work illustrates the hardware and software characteristics of the developed robotic platform, and results obtained with field testing in the real earthquake scenario of Amatrice, Italy. Finally considerations on the experience and feedback provided by civil engineers and architects engaged in the activities are reported and discussed.