Patricio Cruz

Agile Load Transportation : Safe and Efficient Load Manipulation with Aerial Robots

In the past few decades, unmanned aerial vehicles (UAVs) have become promising mobile platforms capable of navigating semiautonomously or autonomously in uncertain environments. The level of autonomy and the flexible technology of these flying robots have rapidly evolved, making it possible to coordinate teams of UAVs in a wide spectrum of tasks. These applications include search and rescue missions; disaster relief operations, such as forest fires [1]; and environmental monitoring and surveillance. In some of these tasks, UAVs work in coordination with other robots, as in robot-assisted inspection at sea [2]. Recently, radio-controlled UAVs carrying radiation sensors and video cameras were used to monitor, diagnose, and evaluate the situation at Japans Fukushima Daiichi nuclear plant facility [3].

Trajectory generation for swing-free maneuvers of a quadrotor with suspended payload: A dynamic programming approach

In this paper, we address the problem of agile swing-free trajectory tracking of a quadrotor with a suspended load. This problem has great practical significance in many UAV applications. However, it has received little attention in the literature so far. Flying with a suspended load can be a very challenging and sometimes hazardous task as the suspended load significantly alters the flight characteristics of the quadrotor. In order to deal with this problem, we propose a technique based on dynamic programming which ensures swing-free trajectory tracking. We start by presenting the mathematical model of a quadrotor with suspended load dynamics and kinematics. A high-level planner is used to provide desired waypoints, and then a dynamic programming approach is used to generate the swing-free trajectory for the quadrotor carrying a suspended load. Effectiveness of this method is demonstrated by numerical simulations and experiments.

Learning swing-free trajectories for UAVs with a suspended load

Attaining autonomous flight is an important task in aerial robotics. Often flight trajectories are not only subject to unknown system dynamics, but also to specific task constraints. This paper presents a motion planning method for generating trajectories with minimal residual oscillations (swing-free) for rotorcraft carrying a suspended loads. We rely on a finite-sampling, batch reinforcement learning algorithm to train the system for a particular load. We find criteria that allow the trained agent to be transferred to a variety of models, state and action spaces and produce a number of different trajectories. Through a combination of simulations and experiments, we demonstrate that the inferred policy is robust to noise and the unmodeled dynamics of the system. The contributions of this work are 1) applying reinforcement learning to solve the problem of finding swing-free trajectories for rotorcraft, 2) designing a problem-specific feature vector for value function approximation, 3) giving sufficient conditions for successful learning transfer to different models, state and action spaces, and 4) verification of the resulting trajectories in both simulation and autonomous control of quadrotors with suspended loads.

A Cooperative Heterogeneous Mobile Wireless Mechatronic System

This paper describes a framework for controlling a heterogeneous wireless robotic network consisting of aerial and ground vehicles. By use of the term heterogeneous, we imply the synergy of multiple robotic platforms characterized by different dynamics and specialized sensing capabilities. Two main scenarios concerning wireless communications are presented: 1) a decentralized connectivity strategy in which a mesh of ground mobile routers swarms in a cluttered environment maintaining communication constraints based on spring-mass virtual physics, potential functions, and routing optimization and 2) an autonomous communications relay in GPS-denied environments via antenna diversity and extremum-seeking SNR optimization. For both scenarios, we validate the proposed methodologies by numerical simulations and experiments. One important feature of our test bed is that it can be used for both indoor and outdoor operations.

A reinforcement learning approach towards autonomous suspended load manipulation using aerial robots

In this paper, we present a problem where a suspended load, carried by a rotorcraft aerial robot, performs trajectory tracking. We want to accomplish this by specifying the reference trajectory for the suspended load only. The aerial robot needs to discover/learn its own trajectory which ensures that the suspended load tracks the reference trajectory. As a solution, we propose a method based on least-square policy iteration (LSPI) which is a type of reinforcement learning algorithm. The proposed method is verified through simulation and experiments.

Automated Aerial Suspended Cargo Delivery through Reinforcement Learning

Abstract Cargo-bearing unmanned aerial vehicles (UAVs) have tremendous potential to assist humans by delivering food, medicine, and other supplies. For time-critical cargo delivery tasks, UAVs need to be able to quickly navigate their environments and deliver suspended payloads with bounded load displacement. As a constraint balancing task for joint UAV-suspended load system dynamics, this task poses a challenge. This article presents a reinforcement learning approach for aerial cargo delivery tasks in environments with static obstacles. We first learn a minimal residual oscillations task policy in obstacle-free environments using a specifically designed feature vector for value function approximation that allows generalization beyond the training domain. The method works in continuous state and discrete action spaces. Since planning for aerial cargo requires very large action space (over 106 actions) that is impractical for learning, we define formal conditions for a class of robotics problems where learning can occur in a simplified problem space and successfully transfer to a broader problem space. Exploiting these guarantees and relying on the discrete action space, we learn the swing-free policy in a subspace several orders of magnitude smaller, and later develop a method for swing-free trajectory planning along a path. As an extension to tasks in environments with static obstacles where the load displacement needs to be bounded throughout the trajectory, sampling-based motion planning generates collision-free paths. Next, a reinforcement learning agent transforms these paths into trajectories that maintain the bound on the load displacement while following the collision-free path in a timely manner. We verify the approach both in simulation and in experiments on a quadrotor with suspended load and verify the methods safety and feasibility through a demonstration where a quadrotor delivers an open container of liquid to a human subject. The contributions of this work are two-fold. First, this article presents a solution to a challenging, and vital problem of planning a constraint-balancing task for an inherently unstable non-linear system in the presence of obstacles. Second, AI and robotics researchers can both benefit from the provided theoretical guarantees of system stability on a class of constraint-balancing tasks that occur in very large action spaces.

Lift of a cable-suspended load by a quadrotor: A hybrid system approach

Autonomous multi-rotor aerial vehicles, specially quadrotors, have become popular platforms for the transportation of cable-suspended loads. Before transporting the load, the lift maneuver is a crucial step that needs to be planed. In order to perform this essential maneuver, we decompose it into simpler hybrid modes which characterize the dynamics of the quadrotor-load system in specific regimes during the maneuver. In this work, we represent the maneuver as a hybrid system and show that it is differentially flat. This property facilitates the generation of a prescribed trajectory and the design of a trajectory tracking controller based on geometric control. Numerical simulations show promising results on the performance of the proposed control architecture.

Autonomous lift of a cable-suspended load by an unmanned aerial robot

In this paper, we address the problem of lifting from the ground a cable-suspended load by a quadrotor aerial vehicle. Furthermore, we consider that the mass of the load is unknown. The lift maneuver is a critical step before proceeding with the transportation of a given cargo. However, it has received little attention in the literature so far. To deal with this problem, we break down the lift maneuver into simpler modes which represent the dynamics of the quadrotor-load system at particular operating regimes. From this decomposition, we obtain a series of waypoints that the aerial vehicle has to reach to accomplish the task. We combine geometric control with a least-squares estimation method to design an adaptive controller that follows a prescribed trajectory planned based on the waypoints. The effectiveness of the proposed control scheme is demonstrated by numerical simulations.

Decentralized identification and control of networks of coupled mobile platforms through adaptive synchronization of chaos

Abstract In this paper, we propose an application of adaptive synchronization of chaos to detect changes in the topology of a mobile robotic network. We assume that the network may evolve in time due to the relative motion of the mobile robots and due to unknown environmental conditions, such as the presence of obstacles in the environment. We consider that each robotic agent is equipped with a chaotic oscillator whose state is propagated to the other robots through wireless communication, with the goal of synchronizing the oscillators. We introduce an adaptive strategy that each agent independently implements to: (i) estimate the net coupling of all the oscillators in its neighborhood and (ii) synchronize the state of the oscillators onto the same time evolution. We show that, by using this strategy, synchronization can be attained and changes in the network topology can be detected. We further consider the possibility of using this information to control the mobile network. We apply our technique to the problem of maintaining a formation between a set of mobile platforms which operate in an inhomogeneous and uncertain environment. We discuss the importance of using chaotic oscillators, and validate our methodology by numerical simulations.

Reinforcement learning for balancing a flying inverted pendulum

The problem of balancing an inverted pendulum on an unmanned aerial vehicle (UAV) has been achieved using linear and nonlinear control approaches. However, to the best of our knowledge, this problem has not been solved using learning methods. On the other hand, the classical inverted pendulum is a common benchmark problem to evaluate learning techniques. In this paper we demonstrate a novel solution to the inverted pendulum problem extended to UAVs, specifically quadrotors. This complex system is underactuated and sensitive to small acceleration changes of the quadrotor. The solution is provided by reinforcement learning (RL), a platform commonly applied to solve nonlinear control problems. We generate a control policy to balance the pendulum using Continuous Action Fitted Value Iteration (CAFVI) [1] which is a RL algorithm for high-dimensional input-spaces. This technique combines learning of both state and state-action value functions in an approximate value iteration setting with continuous inputs. Simulations verify the performance of the generated control policy for varying initial conditions. The results show the control policy is computationally fast enough to be appropriate of real-time control.