Yangsheng Xu

Human action learning via hidden Markov model

To successfully interact with and learn from humans in cooperative modes, robots need a mechanism for recognizing, characterizing, and emulating human skills. In particular, it is our interest to develop the mechanism for recognizing and emulating simple human actions, i.e., a simple activity in a manual operation where no sensory feedback is available. To this end, we have developed a method to model such actions using a hidden Markov model (HMM) representation. We proposed an approach to address two critical problems in action modeling: classifying human action-intent, and learning human skill, for which we elaborated on the method, procedure, and implementation issues in this paper. This work provides a framework for modeling and learning human actions from observations. The approach can be applied to intelligent recognition of manual actions and high-level programming of control input within a supervisory control paradigm, as well as automatic transfer of human skills to robotic systems.

Online, interactive learning of gestures for human/robot interfaces

We have developed a gesture recognition system, based on hidden Markov models, which can interactively recognize gestures and perform online learning of new gestures. In addition, it is able to update its model of a gesture iteratively with each example it recognizes. This system has demonstrated reliable recognition of 14 different gestures after only one or two examples of each. The system is currently interfaced to a Cyberglove for use in recognition of gestures from the sign language alphabet. The system is being implemented as part of an interactive interface for robot teleoperation and programming by example.

Fault diagnosis using support vector machine with an application in sheet metal stamping operations

This paper presents a new method for fault diagnosis using a newly developed method, support vector machine (SVM). First, the basic theory of the SVM is briefly reviewed. Next, a fast implementation algorithm is given. Then the method is applied for the fault diagnosis in sheet metal stamping processes. According to the tests on two different examples, one is a simple blanking and the other is a progressive operation, the new method is very effective. In both cases, its success rate is over 96.5%. In comparison, the success rate of the popular artificial neural network (ANN) is just 93.3%. In addition, the new method requires only few training samples, which is an attractive feature for shop floor applications.

A dual neural network for bi-criteria kinematic control of redundant manipulators

A dual neural network is presented for the bi-criteria kinematic control of redundant manipulators. To diminish the discontinuity of minimum infinity-norm solutions, the kinematic-control problem is formulated in the bi-criteria of the infinity and Euclidean norms. Physical constraints such as joint limits and joint velocity limits are also incorporated simultaneously into the proposed kinematic control scheme. The single-layer dual neural network model with a simple structure is developed for bi-criteria redundant resolution of redundant manipulators subject to robot physical constraints. The dual neural network is shown to be globally convergent to optimal solutions in the bi-criteria sense, and is demonstrated to be effective in controlling the PA10 robot manipulator.

Crowd Density Estimation Using Texture Analysis and Learning

This paper presents an automatic method to detect abnormal crowd density by using texture analysis and learning, which is very important for the intelligent surveillance system in public places. By using the perspective projection model, a series of multi-resolution image cells are generated to make better density estimation in the crowded scene. The cell size is normalized to obtain a uniform representation of texture features. In order to diminish the instability of texture feature measurements, a technique of searching the extrema in the Harris-Laplacian space is also applied. The texture feature vectors are extracted from each input image cell and the support vector machine (SVM) method is utilized to solve the regression problem of calculating the crowd density. Finally, based on the estimated density vectors, the SVM method is used again to solve the classification problem of detecting abnormal density distribution. Experiments on real crowd videos show the effectiveness of the proposed system.

Stochastic similarity for validating human control strategy models

Modeling dynamic human control strategy (HCS), or human skill through learning is becoming an increasingly popular paradigm in many different research areas, such as intelligent vehicle systems, virtual reality, and space robotics. Validating the fidelity of such models requires that we compare the dynamic trajectories generated by the HCS model in the control feedback loop to the original human control data. To this end we have developed a stochastic similarity measure-based on hidden Markov model (HMM) analysis-capable of comparing dynamic, multi-dimensional trajectories. In this paper, we first derive and demonstrate properties of the proposed similarity measure for stochastic systems. We then apply the similarity measure to real-time human driving data by comparing different control strategies for different individuals. Finally, we show that the similarity measure outperforms the more traditional Bayes classifier in correctly grouping driving data from the same individual.

Stabilization and path following of a single wheel robot

We have developed a single wheel, gyroscopically stabilized robot. This is a novel concept for a mobile robot that provides dynamic stability for rapid locomotion. The robot is a sharp-edged wheel actuated by a spinning flywheel for steering and a drive motor for propulsion. The spinning flywheel acts as a gyroscope to stabilize the robot and it can be steered by tilting. This robot is nonholonomic in nature, underactuated and inherently unstable in the lateral direction. In this paper, we first develop a three-dimensional (3-D) nonlinear dynamic model and investigate the dynamic characteristics of the robot. We conduct simulations and real-time experiments to verify the model. Both simulations and experiments show that the flywheel has a significant stabilizing effect on the robot. Then, we can decouple the longitudinal and lateral motions of the robot by linearization. We propose a linear state feedback to stabilize the robot at different lean angles, so as to control the steering velocity of the robot indirectly, because the robot steers only by leaning itself to a predefined angle. For the task of path following, we design a controller for tracking any desired straight line without falling. In the controller, we first design the linear and steering velocities for driving the robot along the desired straight line by controlling the path curvature. We then apply the linear state feedback to stabilize the robot at the predefined lean angle such that the resulting steering velocity of the robot converges to the given steering velocity. This work is a significant step toward fully autonomous control of such a dynamically stable but statically unstable system.

Cooperation control of multiple manipulators with passive joints

This paper studies the problem of modeling and control of multiple cooperative underactuated manipulators handling a rigid object. We reveal holonomic property of such a system by presenting a smooth feedback controller subject to two conditions: 1) there are not fewer active joints than the degrees of freedom of the object; and 2) the Jacobian matrix with respect to passive joints is not singular. This controller is an extension of the PD plus gravity compensation scheme and its asymptotic stability is guaranteed by the LaSalle theorem. Furthermore, we develop a trajectory tracking controller that yields asymptotic convergence of position errors and bounded interaction forces simultaneously. The performance of the proposed controllers has been investigated by simulations on two 6-DOF underactuated manipulators and by experiments on the cooperative underactuated manipulator system developed at CMU.

A single-wheel, gyroscopically stabilized robot

We are developing a novel concept for mobility, and studying fundamental research issues on dynamics and control of the mobile robot. The robot, called Gyrover, is a single-wheel vehicle with an internal gyroscope that provides mechanical stabilization and steering capability. This configuration conveys significant advantages over multi-wheel, statically stable vehicles, including good dynamic stability and insensitivity to attitude disturbances; high manoeuvrability; low rolling resistance; ability to recover from falls; and amphibious capability. In this paper we present the design, analysis and implementation of the robot, as well as the associated research issues and potential applications.

Intelligent shoes for abnormal gait detection

In this paper we introduce a shoe-integrated system for human abnormal gait detection. This intelligent system focuses on detecting the following patterns: normal gait, toe in, toe out, oversupination, and heel walking gait abnormalities. An inertial measurement unit (IMU) consisting of three-dimensional gyroscopes and accelerometers is employed to measure angular velocities and accelerations of the foot. Four force sensing resistors (FSRs) and one bend sensor are installed on the insole of each foot for force and flexion information acquisition. The proposed detection method is mainly based on Principal Component Analysis (PCA) for feature generation and Support Vector Machine (SVM) for multi-pattern classification. In the present study, four subjects tested the shoe-integrated device in outdoor environments. Experimental results demonstrate that the proposed approach is robust and efficient in detecting abnormal gait patterns. Our goal is to provide a cost-effective system for detecting gait abnormalities in order to assist persons with abnormal gaits in the developing of a normal walking pattern in their daily life.