Nicholas R. Gans

Stable Visual Servoing Through Hybrid Switched-System Control

Visual servoing methods are commonly classified as image-based or position-based, depending on whether image features or the camera position define the signal error in the feedback loop of the control law. Choosing one method over the other gives asymptotic stability of the chosen error but surrenders control over the other. This can lead to system failure if feature points are lost or the robot moves to the end of its reachable space. We present a hybrid switched-system visual servo method that utilizes both image-based and position-based control laws. We prove the stability of a specific, state-based switching scheme and present simulated and experimental results.

Homography-Based Control Scheme for Mobile Robots With Nonholonomic and Field-of-View Constraints

In this paper, we present a visual servo controller that effects optimal paths for a nonholonomic differential drive robot with field-of-view constraints imposed by the vision system. The control scheme relies on the computation of homographies between current and goal images, but unlike previous homography-based methods, it does not use the homography to compute estimates of pose parameters. Instead, the control laws are directly expressed in terms of individual entries in the homography matrix. In particular, we develop individual control laws for the three path classes that define the language of optimal paths: rotations, straight-line segments, and logarithmic spirals. These control laws, as well as the switching conditions that define how to sequence path segments, are defined in terms of the entries of homography matrices. The selection of the corresponding control law requires the homography decomposition before starting the navigation. We provide a controllability and stability analysis for our system and give experimental results.

Vision-Based Estimation for Guidance, Navigation, and Control of an Aerial Vehicle

While a Global Positioning System (GPS) is the most widely used sensor modality for aircraft navigation, researchers have been motivated to investigate other navigational sensor modalities because of the desire to operate in GPS denied environments. Due to advances in computer vision and control theory, monocular camera systems have received growing interest as an alternative/collaborative sensor to GPS systems. Cameras can act as navigational sensors by detecting and tracking feature points in an image. Current methods have a limited ability to relate feature points as they enter and leave the camera field of view (FOV). A vision-based position and orientation estimation method for aircraft navigation and control is described. This estimation method accounts for a limited camera FOV by releasing tracked features that are about to leave the FOV and tracking new features. At each time instant that new features are selected for tracking, the previous pose estimate is updated. The vision-based estimation scheme can provide input directly to the vehicle guidance system and autopilot. Simulations are performed wherein the vision-based pose estimation is integrated with a nonlinear flight model of an aircraft. Experimental verification of the pose estimation is performed using the modelled aircraft.

Homography-Based Visual Servo Control With Imperfect Camera Calibration

In this technical note, a robust adaptive uncalibrated visual servo controller is proposed to asymptotically regulate a robot end-effector to a desired pose. A homography-based visual servo control approach is used to address the six degrees-of-freedom regulation problem. A high-gain robust controller is developed to asymptotically stabilize the rotation error, and an adaptive controller is developed to stabilize the translation error while compensating for the unknown depth information and intrinsic camera calibration parameters. A Lyapunov-based analysis is used to examine the stability of the developed controller.

Performance Tests for Visual Servo Control Systems, with Application to Partitioned Approaches to Visual Servo Control

Visual servoing has been a viable method of robot manipulator control for more than a decade. Initial developments involved position-based visual servoing (PBVS), in which the control signal exists in Cartesian space. The younger method, image-based visual servoing (IBVS), has seen considerable development in recent years. PBVS and IBVS offer tradeoffs in performance, and neither can solve all tasks that may confront a robot. In response to these issues, several methods have been devised that partition the control scheme, allowing some motions to be performed in the manner of a PBVS system, while the remaining motions are performed using an IBVS approach. To date, there has been little research that explores the relative strengths and weaknesses of these methods. In this paper we present such an evaluation. We have chosen three recent visual servo approaches for evaluation in addition to the traditional PBVS and IBVS approaches. We posit a set of performance metrics that mea sure quantitatively the performance of a visual servo controller for a specific task. We then evaluate each of the candidate visual servo methods for four canonical tasks with simulations and with experiments in a robotic work cell.

Adaptive Homography-Based Visual Servo Tracking Control via a Quaternion Formulation

In this paper, an adaptive homography-based visual servo tracking controller is developed for the camera-in-hand problem using a quaternion formulation to represent rotation tracking error. The desired trajectory in the tracking problem is encoded by a sequence of images (e.g., a video sequence), and Lyapunov methods are employed to facilitate the control design and the stability analysis. An adaptive estimation law is designed to compensate for the lack of unknown depth information. Experimental results are provided to demonstrate the performance of the visual servo tracking controller.

An asymptotically stable switched system visual controller for eye in hand robots

Visual servoing methods are commonly classified as image based or position based, depending on whether image features or the robot pose is used in the feedback loop of the control law. Choosing one method over the other gives stability in the chosen state but surrenders all control over the other, which can lead to system failure if feature points are lost or the robot moves to the end of its reachable space. We present a hybrid switched system visual servo method that utilizes both image based and position based control laws. Through a switching scheme we present, this method provides asymptotic stability in both the image and pose and prevent system failure.

Keeping Multiple Moving Targets in the Field of View of a Mobile Camera

This study introduces a novel visual servo controller that is designed to control the pose of the camera to keep multiple objects in the field of view (FOV) of a mobile camera. In contrast with other visual servo methods, the control objective is not formulated in terms of a goal pose or a goal image. Rather, a set of underdetermined task functions are developed to regulate the mean and variance of a set of image features. Regulating these task functions inhibits feature points from leaving the camera FOV. An additional task function is used to maintain a high level of motion perceptibility, which ensures that desired feature point velocities can be achieved. These task functions are mapped to camera velocity, which serves as the system input. A proof of stability is presented for tracking three or fewer targets. Experiments of tracking eight or more targets have verified the performance of the proposed method.

A Stable Vision-Based Control Scheme for Nonholonomic Vehicles to Keep a Landmark in the Field of View

Control of wheeled vehicles is a difficult problem due to nonholonomic constraints. This problem is compounded by sensor limitations. A previously developed control scheme for a wheeled robot, which keeps a target in the view of a mounted camera, is one solution to the problem. In this paper, we prove the controllability and stability of the control scheme. We present an implementation of the controller, as well as present the results of simulations and physical experiments.

Visual Servo Velocity and Pose Control of a Wheeled Inverted Pendulum through Partial-Feedback Linearization

Vision-based control of wheeled vehicles is a difficult problem due to nonholonomic constraints on velocities. This is further complicated in the control of vehicles with drift terms and dynamics containing fewer actuators than velocity terms. We explore one such system, the wheeled inverted pendulum, embodied by the Segway. We present two methods of eliminating the effects of nonactuated attitude motions and a novel controller based on partial feedback linearization. This novel controller outperforms a controller based on typical linearization about an equilibrium point