Martin Schätz

Microsoft Kinect Visual and Depth Sensors for Breathing and Heart Rate Analysis

This paper is devoted to a new method of using Microsoft (MS) Kinect sensors for non-contact monitoring of breathing and heart rate estimation to detect possible medical and neurological disorders. Video sequences of facial features and thorax movements are recorded by MS Kinect image, depth and infrared sensors to enable their time analysis in selected regions of interest. The proposed methodology includes the use of computational methods and functional transforms for data selection, as well as their denoising, spectral analysis and visualization, in order to determine specific biomedical features. The results that were obtained verify the correspondence between the evaluation of the breathing frequency that was obtained from the image and infrared data of the mouth area and from the thorax movement that was recorded by the depth sensor. Spectral analysis of the time evolution of the mouth area video frames was also used for heart rate estimation. Results estimated from the image and infrared data of the mouth area were compared with those obtained by contact measurements by Garmin sensors (www.garmin.com). The study proves that simple image and depth sensors can be used to efficiently record biomedical multidimensional data with sufficient accuracy to detect selected biomedical features using specific methods of computational intelligence. The achieved accuracy for non-contact detection of breathing rate was 0.26% and the accuracy of heart rate estimation was 1.47% for the infrared sensor. The following results show how video frames with depth data can be used to differentiate different kinds of breathing. The proposed method enables us to obtain and analyse data for diagnostic purposes in the home environment or during physical activities, enabling efficient human–machine interaction.

Bayesian classification and analysis of gait disorders using image and depth sensors of Microsoft Kinect

This paper presents a novel method of Bayesian gait recognition using Microsoft (MS) Kinect image and depth sensors and skeleton tracking in three-dimensional space. Although video sequences acquired by a complex camera system enable a very precise data analysis, it is possible to use much simpler technical devices to analyze video frames with sufficient accuracy for many applications. The use of the MS Kinect allows a simple 3-D modeling using its image and depth sensors for data acquisition, resulting in a matrix of 640 × 480 elements used for spatial modeling of a moving body. The experimental part of the paper is devoted to the study of three data sets: (i) 18 individuals with Parkinsons disease, (ii) 18 healthy age-matched controls, and (iii) 15 trained young individuals forming the second reference set. The proposed algorithm involves methods for the estimation of the average stride length and gait speed of individuals in these sets. Digital signal processing methods and Bayesian probability classification algorithms are then used for gait feature analysis to recognize individuals suspected of having Parkinsons disease. The results include the estimation of the characteristics of selected gait features for patients with Parkinsons disease and for individuals from the reference sets, presentation of decision boundaries, and comparison of classification efficiency for different features. The achieved accuracy of the probabilistic classification was 94.1%.

The MS kinect image and depth sensors use for gait features detection

Movement disorders, problems with motion and gait stability related to aging form a very intensively studied research area. The paper presents a contribution to these topics through the use of data acquired by motion sensors and namely image and depth sensors of the MS Kinect. While video sequences obtained by complex camera systems can be used for the precise gait features evaluation, it is possible to use much cheaper devices for diagnostic purposes accurate enough in many cases. The experimental part of the study presents video sequences and depth sensors data acquisition for 18 individuals with the Parkinsons disease and 18 healthy age-matched controls using the proposed graphical user interface in the clinical environment. Results presented include the estimation of gait features to distinguish gait disorders and to classify individuals in the early stage of possible serious diseases. The accuracy achieved was higher then 90 % for given sets of individuals.

Feature extraction using MS Kinect and data fusion in analysis of sleep disorders

Non-contact methods for the tracking of breathing have found noticeable interest in research in recent years motivated by the obtrusiveness of the traditional approach to sleep disordered breathing diagnosis. The low-priced Kinect device released by Microsoft has emerged as a possible alternative hardware in the field of subjects monitoring aimed at sleep disorders analysis. In this paper we present a method for the reconstruction of the patients breathing during sleep using the depth maps acquired by Kinect. Preliminary operations of resampling and denoising were performed on the images. A reconstruction of the breathing is then obtained by means of image processing and filtering operations; it is synchronized with the corresponding polysomnographic record, features are extracted from both signals and compared. The strong likeness in the mean of the features extracted from the two records (with mean error of 0.87% in frequency and 9.17% in regularity) supports the view that enhancements of this technique may represent a valid alternative to the present approach to sleep monitoring.

Complex continuous wavelet coherence for EEG microstates detection in insight and calm meditation

Complex continuous wavelet coherence (WTC) can be used for non-stationary signals, such as electroencephalograms. Areas of the WTC with a coherence higher than the calculated optimal threshold were obtained, and the sum of their areas was used as a criterion to differentiate between groups of experienced insight-focused meditators, calm-focused meditators and a control group. This method demonstrated the highest accuracy for the real WTC parts in the frontal region, while for the imaginary parts, the highest accuracy was shown for the frontal occipital pairs of electrodes. In the frontal area, in the broadband frequency, both types of experienced meditators demonstrated an enlargement of the increased coherence areas for the real WTC parts. For the imaginary parts unaffected by the volume conduction and global artefacts, the most significant increase occurred for the frontal occipital pair of electrodes.

Statistical recognition of breathing by MS Kinect depth sensor

Measuring of breathing with contact methods, like respiratory belts, is very uncomfortable for patients and in case of complex sleep analysis, cables from different sensors can substantially affect the quality of the sleep. This paper presents the contactless measuring of breathing using the MS Kinect depth sensor, and it compares the results obtained with records of breathing observed by the flowmetry. The methodological part of the paper is devoted to spectral analysis of data acquired, feature extraction, and their Bayesian classification. The proposed classifier is able to distinguish the Sleep and Wake classes with the accuracy of 100% (cross-validation: 0) for given data. The achieved accuracy of classification into 3 classes (Sleep, Falling Asleep and Wake) is 97% (cross-validation: 0.0248) in the given case.

Video processing and 3D modelling of chest movement using MS Kinect depth sensor

General methods of video processing and three dimensional modelling have a wide range of applications in engineering, archaeology and spacial objects study. The paper is devoted to applications of these methods in biomedicine and neurology using MS Kinect depth sensor for non-contact monitoring of breathing. A special attention is paid to visualization of results and motion mapping over the selected chest area. The proposed methodology applies digital signal processing methods and functional transforms for acquired data de-noising, spectral analysis, and feature selection. Suggested method uses further the local polynomial approximation to detect extremal values of spectral components. The results verify the correspondence between the evaluations of the breathing frequency obtained from the thorax movement recorded by the depth sensor. The study proves that simple depth sensors can be used for non-contact detection of breathing frequency and for the three dimensional modelling of the chest movement. The proposed non-contact method enables to analyse breathing for diagnostic purposes and monitoring in the home environment as a component of assisted living technologies. General methodology studied form a contribution to the use of video sequences or sets of images for spacial objects modelling, their recognition, possible three dimensional printing or analysis of time evolution of their features.

Sleep scoring using polysomnography data features

The paper is devoted to the analysis of multichannel biomedical signals acquired in the sleep laboratory. The data analyzed represent polysomnographic records of (i) 33 healthy individuals, (ii) 25 individuals with sleep apnea, and (iii) 18 individuals with sleep apnea and restless leg syndrome. The initial statistical analysis of the sleep segments points to an increase in the number of Wake stages and the decrease in REM stages with increase in age. The goal of the study is visualization of features associated with sleep stages as specified by an experienced neurologist and in their adaptive classification. The results of the support vector machine classifier are compared with those obtained by the k-nearest neighbors method, decision tree and neural network classification using sigmoidal and Bayesian transfer functions. The achieved accuracy for the classification into two classes (to separate the Wake stage from one of NonREM and REM stages) is between 85.6 and 97.5% for the given set of patients with sleep apnea. The proposed models allow adaptive modification of the model coefficients during the learning process to increase the diagnostic efficiency of sleep disorder analysis, in both the clinical and home environments.

Kinect V2 as a tool for stroke recovery: Pilot study of motion scale monitoring

This paper investigates Kinect device application during rehabilitation of people with an ischemic stroke. There are many similar application using Kinect as a tool during rehabilitation. This paper is focused on measurement of Kinects spatial accuracy and proposition of body states and exercises according to the Motor assessment scale for stroke (MAS). The system observes the whole rehabilitation process and objectively compares ranges of movement during each exercise. Angles between limbs are computed in the skeletal body joints projection to three anatomical planes, which enables a better insight to subject performance. The system is easily implemented with a consumer-grade computer and a low-cost Kinect device. Selected exercises are presented together with the angles evolution, body states recognition and the MAS Scale after the stroke classification.

Face movement analysis with MS Kinect

This experimental study investigated possibility of face symmetry and symmetric face movements evaluation using MS Kinects HD face tracking. The main motivation for this research is facial paralysis or partial loss of muscle control caused by stroke. Evaluation of face symmetry can be used as an indicator of positive improvement of such disability. Precision of face model acquisition of MS Kinect v2 was estimated and two data sets of four facial exercises were recorded. Differences between data sets are evaluated with comparison of left part of face to the right one, with comparison of base frame with face at rest to frames during exercises, and with changes of angles of symmetric points on face. Results show significant differences between both sets, even though face tracking is affected by lightning, distance from camera and position angle of recorded person in view of sensor. Best results are achieved when comparing changes of angles (up to 6°) and differences of in symmetry of face (56% total symmetry in normal set to 13% symmetry in simulated set).