Ahmad M. Qamar

Multimedia interactive therapy environment for children having physical disabilities

In this paper, we present an interactive multimedia environment that can be used to effectively complement the role of a therapist in the process of rehabilitation for disabled children. We use Microsoft Kinect 3D depth sensing camera with the online Second Life virtual world to record rehabilitation exercises performed by a physiotherapist or a disabled child. The exercise session can be played synchronously in Second Life. The physical activities of the users are synchronized with their virtual counterparts in the Second Life. The exercises can be recorded as well and made available for downloading to facilitate offline playback. A disabled child can follow the exercise at home in the absence of the therapist, since the system can provide visual guidance for performing the exercise in the right manner. Using the proposed system, parents at home can also assist the disabled child in performing therapy sessions in the absence of a therapist. Following the suggestions of therapists, the developed prototype can track several gestures of children who have mobility problems. Using a single Kinect device, we can capture high resolution joint movement of the body, without the need for any complicated hardware set up. The initial joint-based angular measurements show promising potential of our prototype to be deployed in real physiotherapy sessions.

Adding Inverse Kinematics for Providing Live Feedback in a Serious Game-Based Rehabilitation System

In this paper, we present a serious game-based framework for providing live feedback to a patient performing a rehabilitation therapy for the purpose of assisting a patient in correct therapeutic steps and obtaining high quality therapy data for further analysis. The game environment uses forward kinematics to receive the live sensory data from two 3D motion tracking sensors and uses inverse kinematics to analyze the sensory data stream in real-time. A subject performs a rehabilitation therapy prescribed by the physician and using both forward and inverse kinematics the system validates the angular and rotational positions of the joints with respect to the correct therapeutic posture and provides live feedback to the subject. As a proof of concept, we have developed an open source web-based framework that can be easily adopted for inhome therapy, without the assistance of a therapist. Finally, we share our initial test result, which is encouraging.

Crowdsourced Multimedia Enhanced Spatio-temporal Constraint Based on-Demand Social Network for Group Mobility

This paper presents a system that enables efficient and scalable real-time user and vehicle discovery using textual, audio and video mechanisms. The system allows users to group together for shared intra-city transportation with the aid of multimedia that helps individuals to 1) find community of common interest (CoCI), 2) locate individual users in a large crowd and 3) locate vehicles for mobility in an efficient and cost effective manner. The system is a pilot project and will be deployed during Hajj 2015 when over three million pilgrims from all over the world visit Makkah, Saudi Arabia.

A GIS-based serious game recommender for online physical therapy

As human-centered interactive technologies, serious games are getting popularity in a variety of fields such as training simulations, health, national defense, and education. To build the best learning experience when designing a serious game, a system requires the integration of accurate spatio-temporal information. Also, there is an increasing need for intelligent medical technologies, which enable patients to live independently at home. This paper introduces a novel e-Health framework that leverages GIS-based serious games for people with disabilities. This framework consists of a spatial map-browsing environment augmented with our newly introduced multi-sensory Natural User Interface. We propose a comprehensive architecture that includes a sensory data manager, a storage layer, an information processing and computational intelligence layer, and a user interface layer. Detailed mathematical modeling as well as mapping methodology to convert different therapy-based hand-gestures into navigational movements within the serious game environment are also presented. Moreover, an Intelligent Game Recommender has been developed for generating optimized navigational routes based on therapeutic gestures. Motion data is stored in a repository throughout the different sessions for offline replaying and advanced analysis; and different indicators are displayed in a live manner. This framework has been tested with Nokia, Google maps, ESRI map, and other maps whereby a subject can visualize and browse the 2D and 3D map of the world through therapy-based gestures. To the best of our knowledge, this is the first GIS-based game re-commender framework for online physical therapy. The prototype has been deployed to a disability center. The obtained results and feedback from therapists and patients are very encouraging.

A Multimedia E-Health Framework Towards An Interactive And Non-Invasive Therapy Monitoring Environment

This paper presents a multimedia e-health framework to conduct therapy sessions by collecting live therapeutic data of patients in a non-invasive way. Using our proposed framework, a therapist can model complex gestures by mapping them to a set of primitive actions and generate high-level therapies. Two inexpensive 3D motion tracking sensors, a Kinect and a Leap, are used to collect motion data of a given subject. Data can then be displayed on screen in a live manner or recorded for offline replaying and analysis. The system incorporates an intelligent authoring tool for therapy design, and produces live plots that show the quality of improvement metrics for a given patient.

Motion-Based Serious Games for Hand Assistive Rehabilitation

Cerebral Palsy, trauma, and strokes are common causes for the loss of hand movements and the decrease in muscle strength for both children and adults. Improving fine motor skills usually involves the synchronization of wrists and fingers by performing appropriate tasks and activities. This demo introduces a novel patient-centered framework for the gamification of hand therapies in order to facilitate and encourage the rehabilitation process. This framework consists of an adaptive therapy-driven 3D environment augmented with our motion-based natural user interface. An intelligent game generator is developed, which translates the patients gestures into navigational movements with therapy-driven goals, while adapting the level of difficulty based on the patient profile and real-time performance. A comprehensive evaluation and clinical-based assessments were conducted in a local children disability center, and highlights of the results are presented.

A therapy-driven gamification framework for hand rehabilitation

Rehabilitative therapy is usually very expensive and confined to specialized rehabilitation centers or hospitals, leading to slower recovery times for corresponding patients. Therefore, there is a high demand for the development of technology-based personalized solutions to guide and encourage patients towards performing online rehabilitation program that can help them live independently at home. This paper introduces an innovative e-health framework that develops adaptive serious games for people with hand disabilities. The aim of this work is to provide a patient-adaptive environment for the gamification of hand therapies in order to facilitate and encourage rehabilitation issues. Theoretical foundations (i.e., therapy and patient models) and algorithms to match therapy-based hand gestures to navigational movements in 3D space within the serious game environment have been developed. A novel game generation module is introduced, which translates those movements into a 3D therapy-driven route on a real-world map and with different levels of difficulty based on the patient profile and capabilities. In order to enrich the user navigation experience, a 3D spatio-temporal validation region is also generated, which tracks and adjusts the patient movements throughout the session. The gaming environment also creates and adds semantics to different types of attractive and repellent objects in space depending on the difficulty level of the game. Relevant benchmarks to assess the patient interaction with the environment along with a usability and performance testing of our framework are introduced to ensure quantitative as well as qualitative improvements. Trial tests in one disability center were conducted with a total number of five subjects, having hand motor controls problems, who used our gamified physiotherapy solution to help us in measuring the usability and users’ satisfaction levels. The obtained results and feedback from therapists and patients are very encouraging.

A Multi-sensory Gesture-Based Login Environment

Logging on to a system using a conventional keyboard may not be feasible in certain environments, such as, in a surgical operation theatre or in an industrial manufacturing facility. We have developed a multi-sensory gesture based login system that allows a user to access secure information using body gestures. The system can be configured to use different types of gestures according to the type of sensors available to the user. We have proposed a simple scheme to represent all alphanumeric characters required for password entry as gestures within the multi-sensory environment. Our scheme is scalable enough to support sensors that detect a large number of gestures to those that can only accept a few. This allows the system to be used in a variety of situations such as usage by disabled persons with limited ability to perform gestures. We are in the midst of deploying our developed system in a clinical environment.

Gamifying hand physical therapy with intelligent 3D navigation

As human-centered interactive technologies, serious games are getting popularity in a variety of fields such as training simulations, health, national defense, and education. To build the best learning experience when designing a serious game, a system requires the integration of accurate spatio-temporal information. Also, there is an increasing need for intelligent medical technologies, which enable patients to live independently at home. This paper introduces a novel e-Health framework that leverages GIS-based serious games for people with disabilities. This framework consists of a spatio-temporal map-browsing environment augmented with our newly introduced multi-sensory natural user interface. We propose a comprehensive architecture that includes a sensory data manager, a storage layer, an information processing and computational intelligence layer, and a user interface layer. Detailed mathematical modeling as well as mapping methodology to convert different therapy-based hand-gestures into navigational movements within the serious game environment are also presented. Moreover, an Intelligent Game Recommender has been developed for generating optimized navigational routes based on therapeutic gestures. Those routes are tailored to the patient preferences and capabilities. Motion data is stored in a repository throughout the different sessions for offline replaying and advanced analysis; and different indicators are displayed in a live manner. To the best of our knowledge, this is the first GIS-based game recommender framework for online physical therapy. The prototype has been deployed to a disability center. The obtained results and feedback from therapists and patients are encouraging.