Yunyi Jia

Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue

This paper describes an approach for a robotic arm to learn new actions through dialogue in a simplified blocks world. In particular, we have developed a threetier action knowledge representation that on one hand, supports the connection between symbolic representations of language and continuous sensorimotor representations of the robot; and on the other hand, supports the application of existing planning algorithms to address novel situations. Our empirical studies have shown that, based on this representation the robot was able to learn and execute basic actions in the blocks world. When a human is engaged in a dialogue to teach the robot new actions, step-by-step instructions lead to better learning performance compared to one-shot instructions.

Design of single-operator-multi-robot teleoperation systems with random communication delay

Multiple robots can be teleoperated by a single operator to provide enhanced capacity and efficiency on accomplishing complicated tasks. The design of this kind of systems is challenging because simultaneously tele-controlling multiple robots exceeds the ability of one single operator. The intelligence and autonomy of the robots need to be integrated into the system. Besides, the communication between the operator and the multi-robot system and the communication among the multiple robots are both subject to communication constraints like time delays and packet losses. This paper designs a non-time based method to realize the single-operator-multi-robot system with random communication delay. The system is designed based on the non-time based teleoperation method and a proposed perceptive coordination method. The random delay problem and the problem of simultaneously tele-controlling multiple robots by a single operator are resolved. Experiments implemented on a multi-robot system illustrated the effectiveness of the design.

Human/robot interaction for human support system by using a mobile manipulator

This paper shows how to use a mobile manipulator as a human support to assist human on accomplishing some complicated tasks. Dynamic model and control of the mobile manipulator are introduced. Two kinds of human support modes are designed by considering singularity, manipulability and safety of the whole system. Experimental results implemented on a mobile manipulator validate the proposed methods.

Teaching Robots New Actions through Natural Language Instructions

Robots often have limited knowledge and need to continuously acquire new knowledge and skills in order to collaborate with its human partners. To address this issue, this paper describes an approach which allows human partners to teach a robot (i.e., a robotic arm) new high-level actions through natural language instructions. In particular, built upon the traditional planning framework, we propose a representation of high-level actions that only consists of the desired goal states rather than step-by-step operations (although these operations may be specified by the human in their instructions). Our empirical results have shown that, given this representation, the robot can reply on automated planning and immediately apply the newly learned action knowledge to perform actions under novel situations.

Development of an omni-directional 3D camera for robot navigation

A novel structured light based omnidirectional 3D camera was developed, which consists of a projector, a camera, and two hyperbolic mirrors. Compared with traditional single-directional 3D sensors, such as kinect, the developed sensor could scan fully 360 degree in its surrounding without any blind area. A panoramic view was fused with dense 3D information could be provided to the operator. Furthermore, A single-shot surface coding method was also developed for such a sensor. Real-time 3D perception for moving objects could be achieved by the proposed coding method. Light rings embedded with different angular frequencies and intensities are emitted from the projector to environment through the hyperbolic mirror. Taking advantage of the epipolar constraints, the invariance of the phase angle is utilized to establish the correspondence and then trangulized to obtain depths information. Compared with existing omnidirectional 3D sensors, the proposed method remained real-time sensing and also obtained dense 3D samples. The proposed 3D camera has been implemented and the experimental results have been analyzed and discussed.

Sensor-based redundancy resolution for a nonholonomic mobile manipulator

Redundant nonholonomic mobile manipulators have wide range applications in civilian and military areas. The high redundancy provides high operation flexibility but also introduces redundancy resolution problems. The existing methods mainly focus on the off-line redundancy resolution, and most of them are based on single objective optimization. However, in order to efficiently accomplish a specific task, the dynamic environment information, task constraints and requirements should be considered. This paper investigates a new method which employs the onboard sensor information to resolve the redundancy online by realizing multiple objectives optimization. The effectiveness of this method is demonstrated by simulation results.

Development and sensitivity analysis of a portable calibration system for joint offset of industrial robot

This paper describes our updated system for industrial robot joint offset calibration. The system consists of an IRB1600 industrial robot, a laser tool attached to the robots end-effector, a portable position-sensitive device (PPD), and a PC based controller. By aiming the laser spot to the center of position-sensitive-detector (PSD) on the PPD with different robot configurations, the developed system ideally implements our proposed calibration method called the virtual line-based single-point constraint approach. However, unlike our previous approach, the calibration method is extended to identify the offset parameters with an uncalibrated laser tool. The position errors of the PPD and the sensitivities of error in the PSD plane to the variation of joint angles are analyzed. Two different robot configuration patterns are compared by implementing the calibration method. Both simulation and real experimental results are consistent with the mathematical analysis. Experimental results with small (10−3−10−2) mean and standard deviation of parameters error verify the effectiveness of both the sensitivity analysis and the developed system.

Saliency-Guided Detection of Unknown Objects in RGB-D Indoor Scenes.

This paper studies the problem of detecting unknown objects within indoor environments in an active and natural manner. The visual saliency scheme utilizing both color and depth cues is proposed to arouse the interests of the machine system for detecting unknown objects at salient positions in a 3D scene. The 3D points at the salient positions are selected as seed points for generating object hypotheses using the 3D shape. We perform multi-class labeling on a Markov random field (MRF) over the voxels of the 3D scene, combining cues from object hypotheses and 3D shape. The results from MRF are further refined by merging the labeled objects, which are spatially connected and have high correlation between color histograms. Quantitative and qualitative evaluations on two benchmark RGB-D datasets illustrate the advantages of the proposed method. The experiments of object detection and manipulation performed on a mobile manipulator validate its effectiveness and practicability in robotic applications.

An online motion planning algorithm for a 7DOF redundant manipulator

For a 7 degree of freedom (7DOF) manipulator which has redundant DOF, traditional numerical algorithm for the inverse kinematic model just provides only one possible solution for desired position and orientation of end-effector which can not fully use the redundancy feature of the 7DOF manipulator. This paper proposes an online motion planning algorithm for a 7DOF manipulator with an analytical inverse kinematic model. This algorithm uses a new constraint named Elbow Angle besides traditional position and orientation matrix constraint to solve the inverse kinematic model. By Using this new constraint, an analytical solution of the inverse kinematic model for this 7DOF manipulator is deduced and the Elbow Angle is autonomously decided online by the environment information. The simulation results demonstrate the high accuracy, fast computation and valuable application of this algorithm.

Visual servoing using non-vector space control theory

Traditional image based visual servoing approaches require feature extraction and tracking during the servoing process. Feature extraction and tracking, however, can be quite difficult in some cases. To eliminate these requirements, this paper presents a visual servoing approach which performs control directly using the intensity information in the image. Considering the image as a set, the approach formulates the servoing problem in the space of sets, which is called non-vector space in this paper. The control theory in this non-vector space is used to formulate the stabilization problem. A controller is designed to stabilize the system around a desired image set. Three types of experiments are carried out on a redundant robotic manipulator to validate the controller. The experimental results verify the correctness of the controller. The approach presented in this paper can be applied to robotic manipulation and navigation.