Javier Varona

Hands-free vision-based interface for computer accessibility

Physically disabled and mentally challenged people are an important part of our society that has not yet received the same opportunities as others in their inclusion in the Information Society. Therefore, it is necessary to develop easily accessible systems for computers to achieve their inclusion within the new technologies. This paper presents a project whose objective is to draw disabled people nearer to new technologies. It presents a vision-based user interface designed to achieve computer accessibility for disabled users with motor impairments. The interface automatically finds the users face and tracks it through time to recognize gestures within the face region in real time. Subsequently, a new information fusion procedure is proposed to acquire data from computer vision algorithms and its results are used to carry out a robust recognition process. Finally, we show how the system is used to replace a conventional mouse device for computer interaction and as a communication system for non-verbal children.

Experiences using a hands-free interface

Hands-free interfaces could be the best choice for Human-Computer Interaction (HCI) for people with physical disabilities that are not capable of using traditional input devices. Once a first prototype is developed in the laboratory taking into account design and usability requirements, real users is what finally categorize an interface as useful or not. Therefore, an evaluation of our interface with users with cerebral palsy and multiple sclerosis has been carried out during a project of 9 months long. This paper presents a vision-based user interface designed to achieve computer accessibility together with the validation and evaluation of its human computer interaction issues such as usability and accessibility.

Understanding dynamic scenes based on human sequence evaluation

In this paper, a Cognitive Vision System (CVS) is presented, which explains the human behaviour of monitored scenes using natural-language texts. This cognitive analysis of human movements recorded in image sequences is here referred to as Human Sequence Evaluation (HSE) which defines a set of transformation modules involved in the automatic generation of semantic descriptions from pixel values. In essence, the trajectories of human agents are obtained to generate textual interpretations of their motion, and also to infer the conceptual relationships of each agent w.r.t. its environment. For this purpose, a human behaviour model based on Situation Graph Trees (SGTs) is considered, which permits both bottom-up (hypothesis generation) and top-down (hypothesis refinement) analysis of dynamic scenes. The resulting system prototype interprets different kinds of behaviour and reports textual descriptions in multiple languages.

Interactive Rehabilitation System for Improvement of Balance Therapies in People With Cerebral Palsy

The present study covers a new experimental system, designed to improve the balance and postural control of adults with cerebral palsy. This system is based on a serious game for balance rehabilitation therapy, designed using the prototype development paradigm and features for rehabilitation with serious games: feedback, adaptability, motivational elements, and monitoring. In addition, the employed interaction technology is based on computer vision because motor rehabilitation consists of body movements that can be recorded, and because vision capture technology is noninvasive and can be used for clients who have difficulties in holding physical devices. Previous research has indicated that serious games help to motivate clients in therapy sessions; however, there remains a paucity of clinical evidence involving functionality. We rigorously evaluated the effects of physiotherapy treatment on balance and gait function of adult subjects with cerebral palsy undergoing our experimental system. A 24-week physiotherapy intervention program was conducted with nine adults from a cerebral palsy center who exercised weekly in 20-min sessions. Findings demonstrated a significant increase in balance and gait function scores resulting in indicators of greater independence for our participating adults. Scores improved from 16 to 21 points in a scale of 28, according to the Tinetti Scale for risk of falls, moving from high fall risk to moderate fall risk. Our promising results indicate that our experimental system is feasible for balance rehabilitation therapy.

Toward natural interaction through visual recognition of body gestures in real-time

In most of the existing human-computer interfaces, enactive knowledge as new natural interaction paradigm has not been fully exploited yet. Recent technological advances have created the possibility to enhance naturally and significantly the interface perception by means of visual inputs, the so-called Vision-Based Interfaces (VBI). In the present paper, we explore the recovery of the users body posture by means of combining robust computer vision techniques and a well known inverse kinematics algorithm in real-time. Specifically, we focus on recognizing the users motions with a particular mean, that is, a body gesture. Defining an appropriate representation of the users body posture based on a temporal parameterization, we apply non-parametric techniques to learn and recognize the users body gestures. This scheme of recognition has been applied to control a computer videogame in real-time to show the viability of the presented approach.

User experience to improve the usability of a vision-based interface

When we develop an input device for users to communicate with computers, we have to take into account that end-users must consider the utilization of the device to be effective, efficient and satisfactory. Users whose expectations are unmet by the interface will tend to abandon it. In this paper we present a vision-based interface for motor-impaired users; a multidisciplinary group developed this interface. The users preferences are a critical issue when selecting an access device; therefore, user requirements should be included in the design. Usability evaluation should be integrated into relevant phases of software development. In order to evaluate the design, we present a process with multiple user studies at different development stages. We describe the combination of a development project and its implementation, with user experience considerations embedded in the process. Finally, we studied the performance of the interface through several tests, paying special attention to satisfaction and fatigue. From our results we observed that although several users found the interface tiring, their satisfaction level was encouraging, suggesting the interface is usable.

Action-specific motion prior for efficient Bayesian 3D human body tracking

In this paper, we aim to reconstruct the 3D motion parameters of a human body model from the known 2D positions of a reduced set of joints in the image plane. Towards this end, an action-specific motion model is trained from a database of real motion-captured performances, and used within a particle filtering framework as a priori knowledge on human motion. First, our dynamic model guides the particles according to similar situations previously learnt. Then, the state space is constrained so only feasible human postures are accepted as valid solutions at each time step. As a result, we are able to track the 3D configuration of the full human body from several cycles of walking motion sequences using only the 2D positions of a very reduced set of joints from lateral or frontal viewpoints.

Evaluation of on-line analytic and numeric inverse kinematics approaches driven by partial vision input

Despite its central role in the constitution of a truly enactive interface, 3D interaction through human full body movement has been hindered by a number of technological and algorithmic factors. Let us mention the cumbersome magnetic equipments, or the underdetermined data set provided by less invasive video-based approaches. In the present paper, we explore the recovery of the full body posture of a standing subject in front of a stereo camera system. The 3D position of the hands, the head and the center of the trunk segment are extracted in real-time and provided to the body posture recovery algorithmic layer. We focus on the comparison between numeric and analytic inverse kinematics approaches in terms of performances and overall quality of the reconstructed body posture. Algorithmic issues arise from the very partial and noisy input and the singularity of the human standing posture. Despite stability concerns, results confirm the pertinence of this approach in this demanding context.

Hands and face tracking for VR applications

In this paper, we present a robust real-time 3D tracking system of human hands and face. This system can be used as a perceptual interface for virtual reality activities in a workbench environment. The main advantage of our system is that the human, placed in front of the virtual reality device, does not need any type of marker or special suit. The system includes a colour segmentation module to detect in real-time the skin-colour pixels present in the images. The results of this skin-colour segmentation will be skin-colour blobs, these are the inputs of a data association module. This module labels the blobs pixels using a set of hypothesis from previous frames. The 2D-tracking results are used for the 3D reconstruction of hands and face in order to obtain the 3D positions of these limbs. Finally, we present several results using the H-ANIM standard to show the systems output performance.

Use of serious games for motivational balance rehabilitation of cerebral palsy patients

Research studies show that serious games help to motivate users in rehabilitation and therapy is better when users are motivated. In this work we experiment with serious games for cerebral palsy patients, who rarely show capacity increases with therapy which causes them demotivation. For this reason, we have implemented balance rehabilitation video games for this group of patients. The video games were developed using the prototype development paradigm, respecting the requirements indicated by physiotherapists and including desirable features for rehabilitation serious games presented in the literature. A set of patients who abandoned therapy last year due to loss of motivation, has tested the video game for a period of 6 months. Whilst using the video game no patients have abandoned therapy, showing the appropriateness of games for this kind of patients.