Filip Malawski

Using Kinect for Facial Expression Recognition under Varying Poses and Illumination

Emotions analysis and recognition by the smartphones with front cameras is a relatively new concept. In this paper we present an algorithm that uses a low resolution 3D sensor for facial expression recognition. The 3D head pose as well as 3D location of the fiducial points are determined using Face Tracking SDK. Tens of the features are automatically selected from a pool determined by all possible line segments between such facial landmarks. We compared correctly classified ratios using features selected by AdaBoost, Lasso and histogram-based algorithms. We compared the classification accuracies obtained both on 3D maps and RGB images. Our results justify the feasibility of low accuracy 3D sensing devices for facial emotion recognition.

Classification of basic footwork in fencing using accelerometer

Analysis and recognition of motion patterns from data acquired by body-worn inertial sensors is an emerging technology in sports. In this paper we propose an effective method for recognition of fencing footwork using a single body-worn accelerometer. We present a challenging dataset consisting of six actions, which were performed by ten persons and repeated ten times by each of them. We propose a segment-based SVM for time-series classification together with a set of informative features. We demonstrate that the method is competitive with 1-NN DTW in terms of classification accuracy. The proposed method achieves classification accuracy slightly better than 70% on the fencing footwork dataset.

Real-Time action detection and analysis in fencing footwork

This paper is devoted to real-time analysis of continuous footwork training routine in fencing. We propose a model-based adaptive filtering algorithm for accurate selection of segments of interest from a velocity signal acquired by the Kinect motion sensor. We remove false positives from the selected segments by extracting dedicated features and applying a SVM classifier. Finally, we compute parameters of the identified lunge actions, which constitute a feedback for the fencers. The proposed methods are evaluated on a dedicated dataset consisting of actions of eight fencers.

System for multimodal data acquisition for human action recognition

Multimodal data is being used more widely for human action recognition nowadays due to the progress of machine learning methods and the development of new types of sensors. The acquisition of the data required by such solutions is often troublesome, and it is difficult to find the proper tools for this process. In this paper, we present a new toolkit for multimodal acquisition. We address and discuss issues concerning the synchronization of data from multiple sensors, the optimization of the initial processing of raw data, and the design of the user interface for efficiently recording large databases. The system was verified in a setup consisting of three types of sensors – a Kinect 2, two PS3Eye cameras, and an accelerometer glove. The accuracy of the synchronization and performance of the initial processing proved to be suitable for human action acquisition and recognition. The system was used for the acquisition of an extensive database of sign language gestures. User feedback indicated the recording process to be efficient, which is also evaluated in the paper. The system is publicly available, both in the form of a standalone application as well as source code, and can be easily customized to any type of sensor setup.

Recognition of action dynamics in fencing using multimodal cues

Abstract Most current approaches to action recognition follow strategies, which permit classification of significantly different actions. However, in some sports disciplines, actions may be distinguished mainly by the dynamics of the motion rather than the trajectory. In this work, we propose a novel approach for recognition of sports actions. The novelty consists in the use of dynamics in the analysis of similar motion patterns. We propose informative motion descriptors based on accelerometric data, skeleton joints features and depth maps, and demonstrate their potential to model the motion dynamics. We show that fusing data from multiple modalities permits better recognition accuracy. We make publicly available a dedicated dataset with fencing footwork samples of ten fencers that consists of depth, skeletal and inertial data of six types of dynamic actions, most of which have similar average trajectories but different dynamics of the motion. We show that on our Fencing Footwork Dataset the proposed method outperforms current state-of-the-art methods for general action recognition.

Compression of image sequences in interactive medical teleconsultations

Interactive medical teleconsultations are an important tool in the modern medical practice. Their applications include remote diagnostics, conferences, workshops and classes for students. In many cases standard medium or low-end machines are employed and the teleconsultation systems must be able to provide high quality of user experience with very limited resources. Particularly problematic are large datasets, consisting of image sequences, which need to be accessed fluently. The main issue is insufficient internal memory, therefore proper compression methods are crucial. However, a scenario where image sequences are kept in a compressed format in the internal memory and decompressed on-the-fly when displayed, is difficult to implement due to performance issues. In this paper we present methods for both lossy and lossless compression of medical image sequences, which require only compatibility with Pixel Shader 2.0 standard, which is present even on relatively old, low-end devices. Based on the evaluation of quality, size reduction and performance, the methods are proved to be suitable and beneficial for the medical teleconsultation applications.

Memory Management in Interactive MedicalTeleconsultation Systems

Medical teleconsultation systems are an important tool in modern medicine, as they enable convenient, real-time collaboration of doctors from remote hospitals, without the need to travel. Due to the large size, efficient handling of medical imaging data in real-time teleconsultations is challenging. Memory management provided by operating systems is based on the virtual memory, which introduces considerable delays. In this paper we propose dedicated methods for memory management, including both memory monitoring and pre-loading of images. Results of the conducted experiments indicate, that the proposed methods can significantly improve the user experience.

Acquisition of databases for facial analysis

This article describes guidelines and recommendations for acquisition of databases for facial analysis. New devices and methods for both face recognition and facial expression recognition are constantly developed. In order to evaluate these devices and methods, dedicated datasets are recorded. Acquisition of a database for facial analysis is not an easy task and requires taking into account multiple issues. Based on our experience with recording databases for facial expression recognition, we provide guidelines regarding the acquisition process. Multiple aspects of such process are discussed in this work, namely selection of sensors and data streams, design and structure of the database, technical aspects, acquisition conditions and design of the user interface. Recommendations how to address these aspects are provided and justified. An acquisition software, designed according to these guidelines, is also discussed. The software was used for recording an extended version of our previous facial expression recognition database and proved to both ensure correct data and be convenient for the recorded subjects.

Holistic approach to design and implementation of a medical teleconsultation workspace

While there are many state-of-the-art approaches to introducing telemedical services in the area of medical imaging, it is hard to point to studies which would address all relevant aspects in a complete and comprehensive manner. In this paper we describe our approach to design and implementation of a universal platform for imaging medicine which is based on our longstanding experience in this area. We claim it is holistic, because, contrary to most of the available studies it addresses all aspects related to creation and utilization of a medical teleconsultation workspace. We present an extensive analysis of requirements, including possible usage scenarios, user needs, organizational and security issues and infrastructure components. We enumerate and analyze multiple usage scenarios related to medical imaging data in treatment, research and educational applications - with typical teleconsultations treated as just one of many possible options. Certain phases common to all these scenarios have been identified, with the resulting classification distinguishing several modes of operation (local vs. remote, collaborative vs. non-interactive etc.). On this basis we propose a system architecture which addresses all of the identified requirements, applying two key concepts: Service Oriented Architecture (SOA) and Virtual Organizations (VO). The SOA paradigm allows us to decompose the functionality of the system into several distinct building blocks, ensuring flexibility and reliability. The VO paradigm defines the cooperation model for all participating healthcare institutions. Our approach is validated by an ICT platform called TeleDICOM II which implements the proposed architecture. All of its main elements are described in detail and cross-checked against the listed requirements. A case study presents the role and usage of the platform in a specific scenario. Finally, our platform is compared with similar systems described into-date studies and available on the market.