David Antón

Exercise Recognition for Kinect-based Telerehabilitation

BACKGROUND An aging population and peoples higher survival to diseases and traumas that leave physical consequences are challenging aspects in the context of an efficient health management. This is why telerehabilitation systems are being developed, to allow monitoring and support of physiotherapy sessions at home, which could reduce healthcare costs while also improving the quality of life of the users. OBJECTIVES Our goal is the development of a Kinect-based algorithm that provides a very accurate real-time monitoring of physical rehabilitation exercises and that also provides a friendly interface oriented both to users and physiotherapists. METHODS The two main constituents of our algorithm are the posture classification method and the exercises recognition method. The exercises consist of series of movements. Each movement is composed of an initial posture, a final posture and the angular trajectories of the limbs involved in the movement. The algorithm was designed and tested with datasets of real movements performed by volunteers. We also explain in the paper how we obtained the optimal values for the trade-off values for posture and trajectory recognition. RESULTS Two relevant aspects of the algorithm were evaluated in our tests, classification accuracy and real-time data processing. We achieved 91.9% accuracy in posture classification and 93.75% accuracy in trajectory recognition. We also checked whether the algorithm was able to process the data in real-time. We found that our algorithm could process more than 20,000 postures per second and all the required trajectory data-series in real-time, which in practice guarantees no perceptible delays. Later on, we carried out two clinical trials with real patients that suffered shoulder disorders. We obtained an exercise monitoring accuracy of 95.16%. CONCLUSIONS We present an exercise recognition algorithm that handles the data provided by Kinect efficiently. The algorithm has been validated in a real scenario where we have verified its suitability. Moreover, we have received a positive feedback from both users and the physiotherapists who took part in the tests.

KiReS: A Kinect-based telerehabilitation system

The goal of this paper is to show the main features of KiReS, a telerehabilitation system based on Kinect for Windows, that offers, for both, users and physiotherapists some specific elements that make it more friendly to them. From the point of view of users, they can see in two 3D avatars how an exercise must be executed and how they execute it respectively. This feature can help them improve exercises performance. Moreover during the rehabilitation session they will always see an informative list that shows the exercises to be done in the session. From the point of view of physiotherapists the system allows them on the one hand, to define customized rehabilitation therapies. That can be done by defining different exercises that combine pre-defined movements. Moreover, they can add tests oriented to specific illnesses so that users themselves evaluate their physical state. On the other hand, they can create new exercises just performing those exercises in front of the system and recording them. Those features, not fully supported by already existing telerehabilitation systems, provide an added value that is well valued by both groups. Moreover, a prototype of KiReS is in operation, and allowed us to test its suitability from the point of view of real time performance as well as from the point of view of usability.

Validation of a Kinect-based telerehabilitation system with total hip replacement patients

The evolving telecommunications industry combined with medical information technology has been proposed as a solution to reduce health care cost and provide remote medical services. This paper aims to validate and show the feasibility and user acceptance of using a telerehabilitation system called Kinect Rehabilitation System (KiReS) in a real scenario, with patients attending repeated rehabilitation sessions after they had a Total Hip Replacement (THR). We present the main features of KiReS, how it was set up in the considered scenario and the experimental results obtained in relation to two different perspectives: patients’ subjective perceptions (gathered through questionnaires) and the accuracy of the performed exercises (by analysing the data captured using KiReS). We made a full deployment of KiReS, defining step by step all the elements of a therapy: postures, movements, exercises and the therapy itself. Seven patients participated in this trial in a total of 19 sessions, and the system recorded 3865 exercise executions. The group showed general support for telerehabilitation and the possibilities that systems such as KiReS bring to physiotherapy treatment.

User experience and interaction performance in 2D/3D telecollaboration

Abstract Affordable 3D cameras, mixed reality headsets, and 3D displays have recently pushed the Augmented Reality (AR) and Virtual Reality (VR) technologies into the consumer market. While these technologies have been adopted in video-game and entertainment industry, the adoption for professional use, such as in industrial and business environment, health-care, and education is still lagging behind. In light of recent advances in mobile communications, AR/VR could pave the way for novel interaction and collaboration of geographically distributed users. Despite the technology being available, majority of communication is still accomplished using traditional video conferencing technology which lacks interactivity, depth perception, and ability to convey non-verbal cues in communication. 3D systems for communication have been proposed to overcome these limitations; however, very few studies looked into the performance and interaction with such technologies. In this paper, we report on a study that examined telecollaboration scenario with three different modalities: 2D video-conferencing, 3D stereoscopic interface, and 3D stereoscopic interface with augmented visual feedback. Twenty participants worked in pairs, assuming the roles of instructor and worker, to remotely interact and perform a set of assembly tasks.

TrhOnt: building an ontology to assist rehabilitation processes

BackgroundOne of the current research efforts in the area of biomedicine is the representation of knowledge in a structured way so that reasoning can be performed on it. More precisely, in the field of physiotherapy, information such as the physiotherapy record of a patient or treatment protocols for specific disorders must be adequately modeled, because they play a relevant role in the management of the evolutionary recovery process of a patient. In this scenario, we introduce TrhOnt, an application ontology that can assist physiotherapists in the management of the patients’ evolution via reasoning supported by semantic technology.MethodsThe ontology was developed following the NeOn Methodology. It integrates knowledge from ontological (e.g. FMA ontology) and non-ontological resources (e.g. a database of movements, exercises and treatment protocols) as well as additional physiotherapy-related knowledge.ResultsWe demonstrate how the ontology fulfills the purpose of providing a reference model for the representation of the physiotherapy-related information that is needed for the whole physiotherapy treatment of patients, since they step for the first time into the physiotherapist’s office, until they are discharged. More specifically, we present the results for each of the intended uses of the ontology listed in the document that specifies its requirements, and show how TrhOnt can answer the competency questions defined within that document. Moreover, we detail the main steps of the process followed to build the TrhOnt ontology in order to facilitate its reproducibility in a similar context. Finally, we show an evaluation of the ontology from different perspectives.ConclusionsTrhOnt has achieved the purpose of allowing for a reasoning process that changes over time according to the patient’s state and performance.

Augmented Telemedicine Platform for Real-Time Remote Medical Consultation

Current telemedicine systems for remote medical consultation are based on decades old video-conferencing technology. Their primary role is to deliver video and voice communication between medical providers and to transmit vital signs of the patient. This technology, however, does not provide the expert physician with the same hands-on experience as when examining a patient in person. Virtual and Augmented Reality (VR and AR) on the other hand have the capacity to enhance the experience and communication between healthcare professionals in geographically distributed locations. By transmitting RGB+D video of the patient, the expert physician can interact with this real-time 3D representation in novel ways. Furthermore, the use of AR technology at the patient side has potential to improve communication by providing clear visual instructions to the caregiver. In this paper, we propose a framework for 3D real-time communication that combines interaction via VR and AR. We demonstrate the capabilities of our framework on a prototype system consisting of a depth camera, projector and 3D display. The system is used to analyze the network performance and data transmission quality of the multimodal streaming in a remote scenario.

A Telerehabilitation System for the Selection, Evaluation and Remote Management of Therapies

Telerehabilitation systems that support physical therapy sessions anywhere can help save healthcare costs while also improving the quality of life of the users that need rehabilitation. The main contribution of this paper is to present, as a whole, all the features supported by the innovative Kinect-based Telerehabilitation System (KiReS). In addition to the functionalities provided by current systems, it handles two new ones that could be incorporated into them, in order to give a step forward towards a new generation of telerehabilitation systems. The knowledge extraction functionality handles knowledge about the physical therapy record of patients and treatment protocols described in an ontology, named TrhOnt, to select the adequate exercises for the rehabilitation of patients. The teleimmersion functionality provides a convenient, effective and user-friendly experience when performing the telerehabilitation, through a two-way real-time multimedia communication. The ontology contains about 2300 classes and 100 properties, and the system allows a reliable transmission of Kinect video depth, audio and skeleton data, being able to adapt to various network conditions. Moreover, the system has been tested with patients who suffered from shoulder disorders or total hip replacement.

Knowledge-Based Telerehabilitation Monitoring

In this paper we describe the main features of an innovative home telerehabilitation system that offers, for both users and physiotherapists, actionable information to gain new insight in the telerehabilitation processes. From the point of view of users, it offers a friendly and immersive exercise interface that shows in two 3D avatars how an exercise must be executed and how the user is executing it respectively. Moreover, during a therapy session, informative elements show up-to-date information to guide and encourage the user. From the point of view of the physiotherapists, the system suggests them appropriate exercises that can be used to define customizable telerehabilitation therapies. Furthermore, another novel contribution of the system is its capacity to transform the raw data collected from a user into information that can help the physiotherapist to improve therapy decision making or the redefinition of existing therapies.