Tiantian Shen

Conferring Robustness to Path-Planning for Image-Based Control

Path-planning has been proposed in visual servoing for reaching the desired location while fulfilling various constraints. Unfortunately, the real trajectory can be significantly different from the reference trajectory due to the presence of uncertainties on the model used, with the consequence that some constraints may not be fulfilled hence leading to a failure of the visual servoing task. This paper proposes a new strategy for addressing this problem, where the idea consists of conferring robustness to the path-planning scheme by considering families of admissible models. In order to obtain these families, uncertainty in the form of random variables is introduced on the available image points and intrinsic parameters. Two families are considered, one by generating a given number of admissible models corresponding to extreme values of the uncertainty, and one by estimating the extreme values of the components of the admissible models. Each model of these families identifies a reference trajectory, which is parametrized by design variables that are common to all the models. The design variables are hence determined by imposing that all the reference trajectories fulfill the required constraints. Discussions on the convergence and robustness of the proposed strategy are provided, in particular showing that the satisfaction of the visibility and workspace constraints for the second family ensures the satisfaction of these constraints for all models bounded by this family. The proposed strategy is illustrated through simulations and experiments.

Visual Servoing Path Planning for Cameras Obeying the Unified Model

Abstract This paper proposes a path planning visual servoing strategy for a class of cameras that includes conventional perspective cameras, fisheye cameras and catadioptric cameras as special cases. Specifically, these cameras are modeled by adopting a unified model recently proposed in the literature and the strategy consists of designing image trajectories for eye-in-hand robotic systems that allow the robot to reach a desired location while satisfying typical visual servoing constraints. To this end, the proposed strategy introduces the projection of the available image features onto a virtual plane and the computation of a feasible image trajectory through polynomial programming. Then, the computed image trajectory is tracked by using an image-based visual servoing controller. Experimental results with a fisheye camera mounted on a 6-d.o.f. robot arm are presented in order to illustrate the proposed strategy.

The Design of Storage System for Digital Airborne Camera Based on SOPC

According to the specified standard of airborne Photogrammetry, digital airborne cameras must have higher performance than ordinary civil cameras, which must shoot with shorter time interval and will generate huge data stream. In this paper, a digital airborne camera is designed and implemented in a single FPGA chip as a SOPC approach. The functions of image acquisition, storage and display are implemented by the hardware logic while the functions of control and communication are implemented by the software running in the soft processor core. With the hardware and software cooperation in the single FPGA, the image whose format is RAW is previewed to a VGA monitor while the raw image data can be transferred to an ATA5 hard disk in Ultra DMA mode 4. In testing, this data storing system can achieve the burst speed to 57MB/s and average speed to 25.43MB/S.

Following a Straight Line in Visual Servoing with Elliptical Projections

The problem of visual servoing to reach the desired location keeping elliptical projections in the camera field of view (FOV) while following a straight line is considered. The proposed approach is representing the whole path with seven polynomials of a path abscise: variables in polynomial coefficients for translational path being zero to represent a minimum path length and for rotational part being adjustable satisfying the FOV limit. The planned elliptical trajectories are tracked by an image-based visual servoing (IBVS) controller. The proposed strategy is verified by a simulational case with a circle and a superposed point, where a traditional IBVS controller directs the camera a detour to the ground, the proposed approach however keeps straight the camera trajectory and also the circle visible. In addition, a six degrees of freedom (6-DoF) articulated arm mounted with a pinhole camera is used to validate the proposed method by taking three Christmas balls as the target.

Visual Servoing Path-Planning with Elliptical Projections

This paper proposes a path planning approach for visual servoing with elliptical projections. 3D primitives like circles and spheres may project onto image plane of a perspective camera as ellipsoids. From these elliptical projections, moment-based features are constructed. Constraints required by the usage of moment-based features will include camera field of view (FOV) limits and occlusion avoidance of all the observed 3D primitives, a straight or an obstacle dodging path, global convergence and etc. We propose to parametrize these constraints into polynomial inequalities in a common path abscise. They share common variables in polynomial coefficients and these variables will be reassigned via a multidimensional nonlinear minimization process until a satisfactory path is obtained. Such a planned path is interpolated into several segments, at the ends of which elliptical projections are tracked by an image-based visual servoing controller. Two synthetic scenarios demonstrate excellent performance of the path-planning algorithm and tracking scheme.

Motion planning from demonstrations and polynomial optimization for visual servoing applications

Vision feedback control techniques are desirable for a wide range of robotics applications due to their robustness to image noise and modeling errors. However in the case of a robot-mounted camera, they encounter difficulties when the camera traverses large displacements. This scenario necessitates continuous visual target feedback during the robot motion, while simultaneously considering the robots self- and external-constraints. Herein, we propose to combine workspace (Cartesian space) path-planning with robot teach-by-demonstration to address the visibility constraint, joint limits and “whole arm” collision avoidance for vision-based control of a robot manipulator. User demonstration data generates safe regions for robot motion with respect to joint limits and potential “whole arm” collisions. Our algorithm uses these safe regions to generate new feasible trajectories under a visibility constraint that achieves the desired view of the target (e.g., a pre-grasping location) in new, undemonstrated locations. Experiments with a 7-DOF articulated arm validate the proposed method.

A visual servoing path-planning strategy for cameras obeying the unified model

Recently, a unified camera model has been introduced in visual control systems in order to describe through a unique mathematical model conventional perspective cameras, fisheye cameras, and catadioptric systems. In this paper, a path-planning strategy for visual servoing is proposed for any camera obeying this unified model. The proposed strategy is based on the projection onto a virtual plane of the available image projections. This has two benefits. First, it allows one to perform camera pose estimation and 3D object reconstruction by using methods for conventional camera that are not valid for other cameras. Second, it allows one to perform image path-planning for multi-constraint satisfaction by using a simplified but equivalent projection model, that in this paper is addressed by introducing polynomial parametrizations of the rotation and translation. The planned image trajectory is hence tracked by using an IBVS controller. The proposed strategy is validated through simulations with image noise and calibration errors typical of real experiments. It is worth remarking that visual servoing path-planning for non conventional perspective cameras has not been proposed yet in the literature.

Inclusion of peripheral correspondences in object and pose estimation for visual servoing path-planning

This article addresses the utilization of peripheral image correspondences obtained in cameras obeying the unified model for path-planning-based visual servoing applications. The proposed approach consists of reprojecting available image correspondences onto an oriented virtual plane, which is designed to carry as many reprojections as possible from correspondences, especially those in periphery region. To realize this, a key transformation matrix is proposed and applied to spherical projections of those correspondences. After this transformation, object and pose estimation are realized via the minimization of algebraic and geometric errors by introducing appropriate parametrization and strategies. The geometric error will be defined either on the virtual plane or on the spherical surface. Minimization of error function defined on the spherical surface provides much more accurate estimation results in object structure than that of other error functions as demonstrated in simulation examples. At last, experiment with a fisheye camera mounted on a robot validates the effectiveness of the proposed method.

Designing parametric linear quadratic regulators for parametric LTI systems via LMIs

ABSTRACT This paper addresses the problem of determining parametric linear quadratic regulators (LQRs) for continuous-time linear-time invariant systems affected by parameters through rational functions. Three situations are considered, where the sought controller has to minimise the best cost, average cost, and worst cost, respectively, over the set of admissible parameters. It is shown that candidates for such controllers can be obtained by solving convex optimisation problems with linear matrix inequality (LMI) constraints. These candidates are guaranteed to approximate arbitrarily well the sought controllers by sufficiently increasing the size of the LMIs. In particular, the candidate that minimises the average cost approximates arbitrarily well the true LQR over the set of admissible parameters. Moreover, conditions for establishing the optimality of the found candidates are provided. Some numerical examples illustrate the proposed methodology.

Optimized vision-based robot motion planning from multiple demonstrations

This paper combines workspace models with optimization techniques to simultaneously address whole-arm collision avoidance, joint limits and camera field of view (FOV) limits for vision-based motion planning of a robot manipulator. A small number of user demonstrations are used to generate a feasible domain over which the whole robot arm can servo without violating joint limits or colliding with obstacles. Our algorithm utilizes these demonstrations to generate new feasible trajectories that keep the target in the camera’s FOV and achieve the desired view of the target (e.g., a pre-grasping location) in new, undemonstrated locations. To fulfill these requirements, a set of control points are selected within the feasible domain. Camera trajectories that traverse these control points are modeled and optimized using either quintic splines (for fast computation) or general polynomials (for better constraint satisfaction). Experiments with a seven degree of freedom articulated arm validate the proposed scheme.