Oscar Osvaldo Sandoval-Gonzalez

Real-Time Gesture Recognition, Evaluation and Feed-Forward Correction of a Multimodal Tai-Chi Platform

This paper presents a multimodal system capable to understand and correct in real-time the movements of Tai-Chi students through the integration of audio-visual-tactile technologies. This platform acts like a virtual teacher that transfers the knowledge of five Tai-Chi movements using feed-back stimuli to compensate the errors committed by a user during the performance of the gesture. The fundamental components of this multimodal interface are the gesture recognition system (using k-means clustering, Probabilistic Neural Networks (PNN) and Finite State Machines (FSM)) and the real-time descriptor of motion which is used to compute and qualify the actual movements performed by the student respect to the movements performed by the master, obtaining several feedbacks and compensating this movement in real-time varying audio-visualtactile parameters of different devices. The experiments of this multimodal platform have confirmed that the quality of the movements performed by the students is improved significantly.

Haptic guidance of Light-Exoskeleton for arm-rehabilitation tasks

Fixed-Base Exoskeleton applications have increased rapidly in the last few years, evidently as part of promising rehabilitation robotic programs of the robotics worldwide community, where in particular Human-Robot-Interaction (HRI) plays an important role in its design and control because they are tightly coupled to human-limbs. Exoskeletons embrace HRI as well as technological and theoretical challenges towards real and effective rehabilitation. In this realm, some questions arise, to name a few, what is the relationship between the exchanged energy between human and exoskeleton? How can we assess rehabilitation factors under HRI philosophy? This paper attempts to establish answers to these questions, which can be embodied into rehabilitation HRI using a Light-Exoskeleton. A compliant haptic guidance scheme for human arm subject to minimum-jerk-trajectories criterion is proposed. Preliminary experimental results provide further insight of a haptic guidance scheme taking into account decisive factors into the HRI such as human pose, haptic guidance control, reaching and tracking tasks, the complexity of the virtual environment, and muscles activity.

Design and Development of a Hand Exoskeleton Robot for Active and Passive Rehabilitation

The present work, which describes the mechatronic design and development of a novel rehabilitation robotic exoskeleton hand, aims to present a solution for neuromusculoskeletal rehabilitation. It presents a full range of motion for all hand phalanges and was specifically designed to carry out position and force-position control for passive and active rehabilitation routines. System integration and preliminary clinical tests are also presented.

Integration of multimodal technologies for a rowing platform

This paper presents the integration of multimodal technologies to measure and transmit different variables and stimuli involved in the human motion analysis, both for the training and for the transfer of users skills achieved by means of a mechatronic rowing platform. Although the mechanical design is the core of this project, this paper describes the integration and interaction of two multimodal systems (the human being and the rowing platform). These systems works together using different sensors and methodologies directly integrated to the human body to obtain detailed information related to the synchronization and correlation among the trajectories, Proximal distance coupling, velocities, muscular activation and muscular force, transversal force in the oars and head position acquired through digital image processing and machine learning techniques. This information is evaluated and used in the rendering section through audio-tactile stimuli for the acceleration learning process

Development of a 3D real time gesture recognition methodology for virtual environment control

In this paper, we propose a real time 3D gesture recognition system that relies on the state based approach. The novelty of this work is the introduction of probabilistic neural networks (PNNs) to characterize the uncertain boundaries of each state. The 3D gestures are modeled as a sequence of states in a configuration space; the number of states and their spatial parameters are calculated by dynamic k-means clustering on the training data of the gesture without temporal information. Gesture recognition is performed using a simple Finite State Machine (FSM), where, each state transition depends only on the output of its corresponding PNN and optionally on its time restrictions (minimum and maximum time permitted in the state). If a recognizer reaches its final state, then it could be said that a gesture is recognized. The approach is illustrated with the implementation of a real time system that recognizes the semantic meaning of seven basic gestures of the Indian Dance, the description of the system and the technologies used, it will be described in detail in the paper.

Digital Image Processing Using LabView

Digital Image processing is a topic of great relevance for practically any project, either for basic arrays of photodetectors or complex robotic systems using artificial vision. It is an interesting topic that offers to multimodal systems the capacity to see and understand their environment in order to interact in a natural and more efficient way. The development of new equipment for high speed image acquisition and with higher resolutions requires a significant effort to develop techniques that process the images in a more efficient way. Besides, medical applications use new image modalities and need algorithms for the interpretation of these images as well as for the registration and fusion of the different modalities, so that the image processing is a productive area for the development of multidisciplinary applications. The aim of this chapter is to present different digital image processing algorithms using LabView and IMAQ vision toolbox. IMAQ vision toolbox presents a complete set of digital image processing and acquisition functions that improve the efficiency of the projects and reduce the programming effort of the users obtaining better results in shorter time. Therefore, the IMAQ vision toolbox of LabView is an interesting tool to analyze in detail and through this chapter it will be presented different theories about digital image processing and different applications in the field of image acquisition, image transformations. This chapter includes in first place the image acquisition and some of the most common operations that can be locally or globally applied, the statistical information generated by the image in a histogram is commented later. Finally, the use of tools allowing to segment or filtrate the image are described making special emphasis in the algorithms of pattern recognition and matching template.

Human forces in hands free interaction: a new paradigm for immersive virtual environments

A recent trend in virtual environments aims at making the user free of devices or limitations in its motion, at the benefit of immersiveness and interactivity. This work introduces an interaction paradigm that goes beyond motion based interaction, by making use of measured real forces exerted by the user in free space. The sensation of touch with virtual object is obtained by vibrotactile stimulation combining virtual contact information with measured forces. In this way the user has an additional dimension of interactivity at the benefit of immersivity, without incurring in the reduction of sense of presence introduced by grounded haptic interfaces. This paradigm is presented and discussed in particular in the context of a boxing training simulation, as one of the possible application scenarios.

Capturing and Training Motor Skills

Abstracting Craft. MIT Press. Norman, D. (1986). User-centered systems design. Hillsdale. Norman, D. (1988). The design of everyday things. New York: Basic Books. M. Chignell, P., & Takeshit, H. (1999). Human-Computer Interaction: The psychology of augmented human behavior. In P. Hancock (A cura di), Human performance and Ergonomics. Academic Press. Oviatt, S. (1993). User Centered Modeling and Evaluation of Multimodal Interfaces. Proceedings of the IEEE , 91 . Qian, G. (2004). A gesture-Driven Multimodal Interactive Dance System. IEEE International Conference on Multimedia and Expo. Sharma, R., Huang, T., & Pavlovic, V. (1998). A Multimodal Framework for Interacting With Virtual Environments. In C. Ntuen, & E. Park (A cura di), Human Interaction With Complex Systems (p. 53-71). Kluwer Academic Publishers. Spitzer, M. (1998). The mind within the net: models of learning, thinking and acting. The MIT press. Tan Chau, P. (2003). Training for physical Tasks in Virtual Environments: Tai Chi.

Approach on a new methodology for skills transfer using a parallel planar robot with visuo-vibrotactile feedback

An approach of a new methodology for skills transfer from machine to human is proposed in this research. This methodology transmits a haptic feed-back using vibrotactile perception to transfer motor skills using a parallel planar robot and virtual reality environments. During the experimentation, the participants tried to learn a specific motion trajectory given by the system. During the process, the system computes the current position and generates a vibrotactile feed-back proportional to the error computed between the actual and the desired position of the motion trajectory. The results of the user studies showed this system can help with learning new skills.

Anthropomorphic robotic system with 6 DOF for space positioning in the virtual reality applications for human machine interaction

This paper presents a spatial hand tracking system using a 6 DOF anthropomorphic robot applied in human machine interaction. The main objective of this mechatronic system is to obtain information about the spatial position of a users hand movements in order to be used like a skills trainer to accelerate the skills transfer from the machine to the human by integrating the laws of physics of virtual objects and adapting different design techniques and use of computer software for three-dimensional virtual reality.