Belinda Lange

FAAST: The Flexible Action and Articulated Skeleton Toolkit

The Flexible Action and Articulated Skeleton Toolkit (FAAST) is middleware to facilitate integration of full-body control with virtual reality applications and video games using OpenNI-compliant depth sensors (currently the PrimeSensor and the Microsoft Kinect). FAAST incorporates a VRPN server for streaming the users skeleton joints over a network, which provides a convenient interface for custom virtual reality applications and games. This body pose information can be used for goals such as realistically puppeting a virtual avatar or controlling an on-screen mouse cursor. Additionally, the toolkit also provides a configurable input emulator that detects human actions and binds them to virtual mouse and keyboard commands, which are sent to the actively selected window. Thus, FAAST can enable natural interaction for existing off-the-shelf video games that were not explicitly developed to support input from motion sensors. The actions and input bindings are configurable at run-time, allowing the user to customize the controls and sensitivity to adjust for individual body types and preferences. In the future, we plan to substantially expand FAASTs action lexicon, provide support for recording and training custom gestures, and incorporate real-time head tracking using computer vision techniques.

Development of an interactive game-based rehabilitation tool for dynamic balance training.

Abstract Conventional physical therapy techniques have been shown to improve balance, mobility, and gait following neurological injury. Treatment involves training patients to transfer weight onto the impaired limb to improve weight shift while standing and walking. Visual biofeedback and force plate systems are often used for treatment of balance and mobility disorders. Researchers have also been exploring the use of video game consoles such as the Nintendo Wii Fit as rehabilitation tools. Case studies have demonstrated that the use of video games may have promise for balance rehabilitation. However, initial usability studies and anecdotal evidence suggest that the current commercial games are not compatible with controlled, specific exercise required to meet therapy goals. Based on focus group data and observations with patients, a game has been developed to specifically target weight shift training using an open source game engine and the Nintendo Wii Fit Balance Board. The prototype underwent initial usability testing with a sample of clinicians and with persons with neurological injury. Overall, feedback was positive, and areas for improvement were identified. This preliminary research provides support for the development of a game that caters specifically to the key requirements of balance rehabilitation.

The Potential of Virtual Reality and Gaming to Assist Successful Aging with Disability

Using the advances in computing power, software and hardware technologies, virtual reality (VR), and gaming applications have the potential to address clinical challenges for a range of disabilities. VR-based games can potentially provide the ability to assess and augment cognitive and motor rehabilitation under a range of stimulus conditions that are not easily controllable and quantifiable in the real world. This article discusses an approach for maximizing function and participation for those aging with and into a disability by combining task-specific training with advances in VR and gaming technologies to enable positive behavioral modifications for independence in the home and community. There is potential for the use of VR and game applications for rehabilitating, maintaining, and enhancing those processes that are affected by aging with and into disability, particularly the need to attain a balance in the interplay between sensorimotor function and cognitive demands and to reap the benefits of task-specific training and regular physical activity and exercise.

Designing informed game-based rehabilitation tasks leveraging advances in virtual reality

Purpose: This paper details a brief history and rationale for the use of virtual reality (VR) technology for clinical research and intervention, and then focuses on game-based VR applications in the area of rehabilitation. An analysis of the match between rehabilitation task requirements and the assets available with VR technology is presented. Key messages and implications: Low-cost camera-based systems capable of tracking user behavior at sufficient levels for game-based virtual rehabilitation activities are currently available for in-home use. Authoring software is now being developed that aims to provide clinicians with a usable toolkit for leveraging this technology. This will facilitate informed professional input on software design, development and application to ensure safe and effective use in the rehabilitation context. Conclusion: The field of rehabilitation generally stands to benefit from the continual advances in VR technology, concomitant system cost reductions and an expanding clinical research literature and knowledge base. Home-based activity within VR systems that are low-cost, easy to deploy and maintain, and meet the requirements for “good” interactive rehabilitation tasks could radically improve users’ access to care, adherence to prescribed training and subsequently enhance functional activity in everyday life in clinical populations. Implications for Rehabilitation Virtual reality (VR) technology has an established track record of success in addressing the therapeutic needs of persons across a range of clinical health conditions. In-home systems for VR rehabilitation are now technologically and pragmatically feasible, but it will require informed professional input on software design, development and application to ensure safe and effective use. New tools are being created that will allow clinicians without programming expertise to build game-based VR tasks and this will serve to drive advances in rehabilitation interventions.

Virtual Reality Goes to War: A Brief Review of the Future of Military Behavioral Healthcare

Numerous reports indicate that the incidence of posttraumatic stress disorder (PTSD) in returning OEF/OIF military personnel is creating a significant healthcare challenge. These findings have served to motivate research on how to better develop and disseminate evidence-based treatments for PTSD. Virtual Reality delivered exposure therapy for PTSD has been previously used with reports of positive outcomes. This article details how virtual reality applications are being designed and implemented across various points in the military deployment cycle to prevent, identify and treat combat-related PTSD in OIF/OEF Service Members and Veterans. The summarized projects in these areas have been developed at the University of Southern California Institute for Creative Technologies, a U.S. Army University Affiliated Research Center, and this paper will detail efforts to use virtual reality to deliver exposure therapy, assess PTSD and cognitive function and provide stress resilience training prior to deployment.

Interactive game-based rehabilitation using the Microsoft Kinect

Using video games in rehabilitation settings has the potential to provide patients with fun and motivating exercise tools. Within the Medical VR and MxR groups at the USC Institute for Creative Technologies, we have been leveraging the technology of the Microsoft Kinect 3D depth-sensing camera. Our Kinect-based rehabilitation game “JewelMine” consists of a set of static balance training exercises which encourage the players to reach out of their base of support. We plan to demonstrate a sophisticated post-session analysis tool and several content themes which can be changed dynamically during a therapy session.

An Intelligent Virtual Human System For Providing Healthcare Information And Support

Abstract Over the last 15 years, a virtual revolution has taken place in the use of Virtual Reality simulation technology for clinical purposes. Shifts in the social and scientific landscape have now set the stage for the next major movement in Clinical Virtual Reality with the “birth” of intelligent virtual humans. Seminal research and development has appeared in the creation of highly interactive, artificially intelligent and natural language capable virtual human agents that can engage real human users in a credible fashion. No longer at the level of a prop to add context or minimal faux interaction in a virtual world, virtual humans can be designed to perceive and act in a 3D virtual world, engage in spoken dialogs with real users and can be capable of exhibiting human-like emotional reactions. This paper will present an overview of the SimCoach project that aims to develop virtual human support agents to serve as online guides for promoting access to psychological healthcare information and for assisting military personnel and family members in breaking down barriers to initiating care. The SimCoach experience is being designed to attract and engage military Service Members, Veterans and their significant others who might not otherwise seek help with a live healthcare provider. It is expected that this experience will motivate users to take the first step – to empower themselves to seek advice and information regarding their healthcare and general personal welfare and encourage them to take the next step towards seeking other, more formal resources if needed.

Virtual Reality and Interactive Digital Game Technology: New Tools to Address Obesity and Diabetes

The convergence of the exponential advances in virtual reality (VR)-enabling technologies with a growing body of clinical research and experience has fueled the evolution of the discipline of clinical VR. This article begins with a brief overview of methods for producing and delivering VR environments that can be accessed by users for a range of clinical health conditions. Interactive digital games and new forms of natural movement-based interface devices are also discussed in the context of the emerging area of exergaming, along with some of the early results from studies of energy expenditure during the use of these systems. While these results suggest that playing currently available active exergames uses significantly more energy than sedentary activities and is equivalent to a brisk walk, these activities do not reach the level of intensity that would match playing the actual sport, nor do they deliver the recommended daily amount of exercise for children. However, these results provide some support for the use of digital exergames using the current state of technology as a complement to, rather than a replacement, for regular exercise. This may change in the future as new advances in novel full-body interaction systems for providing vigorous interaction with digital games are expected to drive the creation of engaging, low-cost interactive game-based applications designed to increase exercise participation in persons at risk for obesity.

Special Section on Touching the 3rd Dimension: Adapting user interfaces for gestural interaction with the flexible action and articulated skeleton toolkit

We present the Flexible Action and Articulated Skeleton Toolkit (FAAST), a middleware software framework for integrating full-body interaction with virtual environments, video games, and other user interfaces. This toolkit provides a complete end-to-end solution that includes a graphical user interface for custom gesture creation, sensor configuration, skeletal tracking, action recognition, and a variety of output mechanisms to control third party applications, allowing virtually any PC application to be repurposed for gestural control even if it does not explicit support input from motion sensors. To facilitate intuitive and transparent gesture design, we define a syntax for representing human gestures using rule sets that correspond to the basic spatial and temporal components of an action. These individual rules form primitives that, although conceptually simple on their own, can be combined both simultaneously and in sequence to form sophisticated gestural interactions. In addition to presenting the system architecture and our approach for representing and designing gestural interactions, we also describe two case studies that evaluated the use of FAAST for controlling first-person video games and improving the accessibility of computing interfaces for individuals with motor impairments. Thus, this work represents an important step toward making gestural interaction more accessible for practitioners, researchers, and hobbyists alike.

Virtual Reality Applications to Address the Wounds of War

Healio.com/Psychiatry | 123 Numerous reports indicate that the incidence of posttraumatic stress disorder (PTSD) in returning Operation Enduring Freedom/ Operation Iraqi Freedom (OEF/OIF) military personnel is creating a significant behavioral health care challenge. These findings have served to motivate research on how to better develop and disseminate evidence-based treatments for PTSD. This article details how virtual reality applications are being designed and implemented across various points in the military deployment cycle to prevent, identify, and treat combat-related PTSD in OEF/ OIF service members and veterans. The summarized projects in these areas have been developed at the University of Southern California Institute for Creative Technologies (USC ICT), a US Army University Affiliated Research Center, and will detail efforts to use virtual reality to deliver exposure therapy and provide stress resilience training prior to deployVirtual Reality Applications to Address the Wounds of War