Marc D. Killpack

Reaching in clutter with whole-arm tactile sensing

Clutter creates challenges for robot manipulation, including a lack of non-contact trajectories and reduced visibility for line-of-sight sensors. We demonstrate that robots can use whole-arm tactile sensing to perceive clutter and maneuver within it, while keeping contact forces low. We first present our approach to manipulation, which emphasizes the benefits of making contact across the entire manipulator and assumes the manipulator has low-stiffness actuation and tactile sensing across its entire surface. We then present a novel controller that exploits these assumptions. The controller only requires haptic sensing, handles multiple contacts, and does not need an explicit model of the environment prior to contact. It uses model predictive control with a time horizon of length one and a linear quasi-static mechanical model. In our experiments, the controller enabled a real robot and a simulated robot to reach goal locations in a variety of environments, including artificial foliage, a cinder block, and randomly generated clutter, while keeping contact forces low. While reaching, the robots performed maneuvers that included bending objects, compressing objects, sliding objects, and pivoting around objects. In simulation, whole-arm tactile sensing also outperformed per-link force-torque sensing in moderate clutter, with the relative benefits increasing with the amount of clutter.

Tactile sensing over articulated joints with stretchable sensors

Biological organisms benefit from tactile sensing across the entire surfaces of their bodies. Robots may also be able to benefit from this type of sensing, but fully covering a robot with robust and capable tactile sensors entails numerous challenges. To date, most tactile sensors for robots have been used to cover rigid surfaces. In this paper, we focus on the challenge of tactile sensing across articulated joints, which requires sensing across a surface whose geometry varies over time. We first demonstrate the importance of sensing across joints by simulating a planar arm reaching in clutter and finding the frequency of contact at the joints. We then present a simple model of how much a tactile sensor would need to stretch in order to cover a 2 degree-of-freedom (DoF) wrist joint. Next, we describe and characterize a new tactile sensor made with stretchable fabrics. Finally, we present results for a stretchable sleeve with 25 tactile sensors that covers the forearm, 2 DoF wrist, and end effector of a humanoid robot. This sleeve enabled the robot to reach a target in instrumented clutter and reduce contact forces.

Visual odometry and control for an omnidirectional mobile robot with a downward-facing camera

An omnidirectional Mecanum base allows for more flexible mobile manipulation. However, slipping of the Mecanum wheels results in poor dead-reckoning estimates from wheel encoders, limiting the accuracy and overall utility of this type of base. We present a system with a downward-facing camera and light ring to provide robust visual odometry estimates. We mounted the system under the robot which allows it to operate in conditions such as large crowds or low ambient lighting. We demonstrate that the visual odometry estimates are sufficient to generate closed-loop PID (Proportional Integral Derivative) and LQR (Linear Quadratic Regulator) controllers for motion control in three different scenarios: waypoint tracking, small disturbance rejection, and sideways motion. We report quantitative measurements that demonstrate superior control performance when using visual odometry compared to wheel encoders. Finally, we show that this system provides high-fidelity odometry estimates and is able to compensate for wheel slip on a four-wheeled omnidirectional mobile robot base.

A New Soft Robot Control Method: Using Model Predictive Control for a Pneumatically Actuated Humanoid

Traditional rigid robots, such as those used in manufacturing, have been effective at precise, accurate, rapid motions in well-structured environments for many decades now. However, they operate largely behind cages due to the danger of injury when moving in close proximity to people. A significant and recent shift in robotics involves trading rigid links and rigid actuators for soft, deformable links and compliant actuators. These soft robots generally have lower inertia and avoid many of the problems caused by the high effective inertia resulting from the high gear ratios necessary for rigid robots.

Control of a pneumatically actuated, fully inflatable, fabric-based, humanoid robot

Although humanoid robots take the form of humans, these robots often approach manipulating the world in a very different way than humans. For example, many humanoid robots require precise position control and geometric models to interact successfully with the world. Humanoid robots also often avoid making contact with the world unless the contact can be well modeled. In this work, we present preliminary results on soft robot platforms that can change the way humanoid robots interact with humans and human environments. We present preliminary control methods and testing on fully inflatable, pneumatically actuated, soft robots. We first show that model predictive control (MPC) and linear quadratic regulation (LQR) are sufficient for position control of a single joint with one degree of freedom. We also demonstrate MPC and LQR as methods of control for an inflatable humanoid robot on one arm using five degrees of freedom. Our initial development for multi-joint control is based on the methods developed for the single degree of freedom platform. Using the MPC controller with joint space commands, a task of picking up a board from a chair and placing it in a box was successful eight out of ten times. Our models and control methods will allow for a new type of humanoid robots that are well suited to interacting more safely and naturally in human environments.

Simultaneous position and stiffness control for an inflatable soft robot

Soft robot research has led to the development of platforms that should allow for better performance when working in uncertain or dynamic environments. The potential improvement in performance of these platforms ranges from mechanical robustness to high forces, to applying lower incidental contact forces in uncertain situations. However, the promise of these platforms is limited by the difficulty of controlling them. In this paper, we present preliminary results on simultaneously controlling stiffness and position for a pneumatically actuated soft robot. Improving on our prior work, we show that by including the pressure in our soft robot actuation chambers as state variables we can improve our average rise time by up to 137%, settling time by 119%, and overshoot by 853%. In addition to these improvements, we can now control both joint position and stiffness simultaneously. This performance improvement comes from using Model Predictive Control running at 300 Hz with improved dynamic models of the soft robot. High performance control of soft robot joints, such as the joint presented in this paper, will enable a wide range of robot applications that were previously difficult or impossible due to the rigid nature of traditional robot linkages and actuation schemes.

Fast reaching in clutter while regulating forces using model predictive control

Moving a robot arm quickly in cluttered and unmodeled workspaces can be difficult because of the inherent risk of high impact forces. Additionally, compliance by itself is not enough to limit contact forces due to multi-contact phenomena (jamming, etc.). The work in this paper extends our previous research on manipulation in cluttered environments by explicitly modeling robot arm dynamics and using model predictive control (MPC) with whole-arm tactile sensing to improve the speed and force control. We first derive discrete-time dynamic equations of motion that we use for MPC. Then we formulate a multi-time step model predictive controller that uses this dynamic model. These changes allow us to control contact forces while increasing overall end effector speed. We also describe a constraint that regulates joint velocities in order to mitigate unexpected impact forces while reaching to a goal. We present results using tests from a simulated three link planar arm that is representative of the kinematics and mass of an average males torso, shoulder and elbow joints reaching in high and low clutter scenarios. These results show that our controller allows the arm to reach a goal up to twice as fast as our previous work, while still controlling the contact forces to be near a user-defined threshold.

Model predictive control for fast reaching in clutter

A key challenge for haptically reaching in dense clutter is the frequent contact that can occur between the robot’s arm and the environment. We have previously used single-time-step model predictive control (MPC) to enable a robot to slowly reach into dense clutter using a quasistatic mechanical model. Rapid reaching in clutter would be desirable, but entails additional challenges due to dynamic phenomena that can lead to higher forces from impacts and other types of contact. In this paper, we present a multi-time-step MPC formulation that enables a robot to rapidly reach a target position in dense clutter, while regulating whole-body contact forces to be below a given threshold. Our controller models the dynamics of the arm in contact with the environment in order to predict how contact forces will change and how the robot’s end effector will move. It also models how joint velocities will influence potential impact forces. At each time step, our controller uses linear models to generate a convex optimization problem that it can solve efficiently. Through tens of thousands of trials in simulation, we show that with our dynamic MPC a simulated robot can, on average, reach goals 1.4 to 2 times faster than our previous controller, while attaining comparable success rates and fewer occurrences of high forces. We also conducted trials using a real 7 degree-of-freedom (DoF) humanoid robot arm with whole-arm tactile sensing. Our controller enabled the robot to rapidly reach target positions in dense artificial foliage while keeping contact forces low.

Comparing Model Predictive Control and input shaping for improved response of low-impedance robots

With an increasing number of robots that can exhibit compliant behavior for safety in operating near humans (either through passive components or active control), additional methods for controlling these robots are needed. In particular, robot arms with low impedance can be safer for working in delicate environments if the effects of dealing with an underdamped robot system can be mitigated to improve performance. This paper focuses on comparing methods that allow a seven degree of freedom Series Elastic Actuator arm to operate with very low impedance while mitigating unwanted oscillation at the end effector. We show that by implementing feedback linearizion in conjunction with input shaping we can reduce residual oscillation for a seven degree of freedom robot arm. We also show that a Cartesian Model Predictive Controller (MPC) is able to significantly reduce residual oscillations while maintaining compliance. Comparing these two methods shows that for our tests for large displacements, MPC has a maximum overshoot of only 0.26% in the worst case where input shaping has at least 5.80% overshoot even in the best case. In addition, despite the fact that MPC is a feedback controller (unlike the open-loop input shaping method), it is still able to maintain compliance at the joints and end effector where we estimated MPC to exhibit a stiffness of 234 N/m as compared to the nominal low impedance controller with a stiffness of 262 N/m. Similar to input shaping (which is a command generation method), MPC is able to generate these commands without any slewing or path planning.

Whole-arm tactile sensing for beneficial and acceptable contact during robotic assistance

Many assistive tasks involve manipulation near the care-receivers body, including self-care tasks such as dressing, feeding, and personal hygiene. A robot can provide assistance with these tasks by moving its end effector to poses near the care-receivers body. However, perceiving and maneuvering around the care-receivers body can be challenging due to a variety of issues, including convoluted geometry, compliant materials, body motion, hidden surfaces, and the object upon which the body is resting (e.g., a wheelchair or bed). Using geometric simulations, we first show that an assistive robot can achieve a much larger percentage of end-effector poses near the care-receivers body if its arm is allowed to make contact. Second, we present a novel system with a custom controller and whole-arm tactile sensor array that enables a Willow Garage PR2 to regulate contact forces across its entire arm while moving its end effector to a commanded pose. We then describe tests with two people with motor impairments, one of whom used the system to grasp and pull a blanket over himself and to grab a cloth and wipe his face, all while in bed at his home. Finally, we describe a study with eight able-bodied users in which they used the system to place objects near their bodies. On average, users perceived the system to be safe and comfortable, even though substantial contact occurred between the robots arm and the users body.