Robert Paolini

Extrinsic dexterity: In-hand manipulation with external forces

“In-hand manipulation” is the ability to reposition an object in the hand, for example when adjusting the grasp of a hammer before hammering a nail. The common approach to in-hand manipulation with robotic hands, known as dexterous manipulation [1], is to hold an object within the fingertips of the hand and wiggle the fingers, or walk them along the objects surface. Dexterous manipulation, however, is just one of the many techniques available to the robot. The robot can also roll the object in the hand by using gravity, or adjust the objects pose by pressing it against a surface, or if fast enough, it can even toss the object in the air and catch it in a different pose. All these techniques have one thing in common: they rely on resources extrinsic to the hand, either gravity, external contacts or dynamic arm motions. We refer to them as “extrinsic dexterity”. In this paper we study extrinsic dexterity in the context of regrasp operations, for example when switching from a power to a precision grasp, and we demonstrate that even simple grippers are capable of ample in-hand manipulation. We develop twelve regrasp actions, all open-loop and hand-scripted, and evaluate their effectiveness with over 1200 trials of regrasps and sequences of regrasps, for three different objects (see video [2]). The long-term goal of this work is to develop a general repertoire of these behaviors, and to understand how such a repertoire might eventually constitute a general-purpose in-hand manipulation capability.

A convex polynomial force-motion model for planar sliding: Identification and application

We propose a polynomial force-motion model for planar sliding. The set of generalized friction loads is the 1-sublevel set of a polynomial whose gradient directions correspond to generalized velocities. Additionally, the polynomial is confined to be convex even-degree homogeneous in order to obey the maximum work inequality, symmetry, shape invariance in scale, and fast invertibility. We present a simple and statistically-efficient model identification procedure using a sum-of-squares convex relaxation. Simulation and robotic experiments validate the accuracy and efficiency of our approach. We also show practical applications of our model including stable pushing of objects and free sliding dynamic simulations.

A data-driven statistical framework for post-grasp manipulation

Grasping an object is usually only an intermediate goal for a robotic manipulator. To finish the task, the robot needs to know where the object is in its hand and what action to execute. This paper presents a general statistical framework to address these problems. Given a novel object, the robot learns a statistical model of grasp state conditioned on sensor values. The robot also builds a statistical model of the requirements for a successful execution of the task in terms of uncertainty in the state of the grasp. Both of these models are constructed by offline experiments. The online process then grasps objects and chooses actions to maximize likelihood of success. This paper describes the framework in detail, and demonstrates its effectiveness experimentally in placing, dropping, and insertion tasks. To construct statistical models, the robot performed over 8,000 grasp trials, and over 1,000 trials each of placing, dropping, and insertion.

A general framework for open-loop pivoting

Pivoting is the rotation of an object between two fingers using gravity and inertial forces to impart angular momentum. We present an analysis of the mechanics of pivoting and a framework for planning and execution. Extrinsic dexterity was defined by Chavan-Dafle et al. [1] as the use of external forces, such as gravity and inertial forces in post grasp manipulation. We analyze one such regrasp termed “pivoting” by Rao et al. [2]. We find a grasp and arm trajectory which can rotate an object between stable poses, if any. We demonstrate an implementation of pivoting with an ABB industrial arm and a two fingered gripper.

Regrasping objects using extrinsic dexterity

This video presents the application of Extrinsic Dexterity to change the pose of an object in the hand, i.e., to regrasp the object. Gravity, inertia, arm motions and external contacts can be exploited to manipulate an object in the hand. As such dexterity does not depend solely on the intrinsic capability of the hand, but rather is derived from external resources, we call it as “Extrinsic Dexterity”. The video showcases a repertoire of regrasps developed for a simple gripper and presents one of the sequences regrasps designed to explore border manipulation capability by connecting different regrasps.

A Probabilistic Planning Framework for Planar Grasping Under Uncertainty

How can a robot design a sequence of grasping actions that will succeed despite the presence of bounded state uncertainty and an inherently stochastic system? In this letter, we propose a probabilistic algorithm that generates sequential actions to iteratively reduce uncertainty until object pose is uniquely known (subject to symmetry). The plans assume encoder feedback that gives a geometric partition of the post-grasp configuration space based on contact conditions. An offline planning tree is generated by interleaving computationally tractable open-loop action sequence search and feedback state estimation with particle filtering. To speed up planning, we use learned approximate forward motion models, sensor models, and collision detectors. We demonstrate the efficacy of our algorithm on robotic experiments with more than 3000 grasp sequences using different object shapes, press-ure distributions, and gripper materials where the uncertainty region is comparable to the size of the object in translation and with no information about orientation.

A convex polynomial model for planar sliding mechanics: theory, application, and experimental validation

We propose a polynomial model for planar sliding mechanics. For the force–motion mapping, we treat the set of generalized friction loads as the 1-sublevel set of a polynomial whose gradient directions correspond to generalized velocities. The polynomial is confined to be convex even-degree homogeneous in order to obey the maximum work inequality, symmetry, shape invariance in scale, and fast invertibility. We present a simple and statistically efficient model identification procedure using a sum-of-squares convex relaxation. We then derive the kinematic contact model that resolves the contact modes and instantaneous object motion given a position controlled manipulator action. The inherently stochastic object-to-surface friction distributions are modeled by sampling polynomial parameters from distributions that preserve sum-of-squares convexity. Thanks to the model smoothness, the mechanics of patch contact is captured while being computationally efficient without mode selection at support points. Simulation and robotic experiments on pushing and grasping validate the accuracy and efficiency of our approach.

Data-driven statistical modeling of a cube regrasp

Regrasping is the process of adjusting the position and orientation of an object in ones hand. The study of robotic regrasping has generally been limited to use of theoretical analytical models and cases with little uncertainty. Analytical models and simulations have so far proven unable to capture the complexity of the real world. Empirical statistical models are more promising, but collecting good data is difficult. In this paper, we collect data from 3300 robot regrasps, and use this data to learn two probability functions: 1) The probability that the object is still in the robots hand after a regrasp action; and 2) The probability distribution of the object pose after the regrasp given that the object is still grasped. Both of these functions are learned using kernel density estimation with a similarity metric over object pose. We show that our data-driven models achieve comparable accuracy to a geometric model and an off-the-shelf simulator in classification and prediction tasks, while also enabling us to predict probability distributions.