Davor Kukolja

Physiology-driven adaptive virtual reality stimulation for prevention and treatment of stress related disorders

The significant proportion of severe psychological problems related to intensive stress in recent large peacekeeping operations underscores the importance of effective methods for strengthening the prevention and treatment of stress-related disorders. Adaptive control of virtual reality (VR) stimulation presented in this work, based on estimation of the persons emotional state from physiological signals, may enhance existing stress inoculation training (SIT). Physiology-driven adaptive VR stimulation can tailor the progress of stressful stimuli delivery to the physiological characteristics of each individual, which is indicated for improvement in stress resistance. Following an overview of physiology-driven adaptive VR stimulation, its major functional subsystems are described in more detail. A specific algorithm of stimuli delivery applicable to SIT is outlined.

Comparative analysis of emotion estimation methods based on physiological measurements for real-time applications

In order to improve intelligent Human-Computer Interaction it is important to create a personalized adaptive emotion estimator that is able to learn over time emotional response idiosyncrasies of individual person and thus enhance estimation accuracy. This paper, with the aim of identifying preferable methods for such a concept, presents an experiment-based comparative study of seven feature reduction and seven machine learning methods commonly used for emotion estimation based on physiological signals. The analysis was performed on data obtained in an emotion elicitation experiment involving 14 participants. Specific discrete emotions were targeted with stimuli from the International Affective Picture System database. The experiment was necessary to achieve the uniformity in the various aspects of emotion elicitation, data processing, feature calculation, self-reporting procedures and estimation evaluation, in order to avoid inconsistency problems that arise when results from studies that use different emotion-related databases are mutually compared. The results of the performed experiment indicate that the combination of a multilayer perceptron (MLP) with sequential floating forward selection (SFFS) exhibited the highest accuracy in discrete emotion classification based on physiological features calculated from ECG, respiration, skin conductance and skin temperature. Using leave-one-session-out crossvalidation method, 60.3% accuracy in classification of 5 discrete emotions (sadness, disgust, fear, happiness and neutral) was obtained. In order to identify which methods may be the most suitable for real-time estimator adaptation, execution and learning times of emotion estimators were also comparatively analyzed. Based on this analysis, preferred feature reduction method for real-time estimator adaptation was minimum redundancy - maximum relevance (mRMR), which was the fastest approach in terms of combined execution and learning time, as well as the second best in accuracy, after SFFS. In combination with mRMR, highest accuracies were achieved by k-nearest neighbor (kNN) and MLP with negligible difference (50.33% versus 50.54%); however, mRMR+kNN is preferable option for real-time estimator adaptation due to considerably lower combined execution and learning time of kNN versus MLP.

Real-Time Emotional State Estimator for Adaptive Virtual Reality Stimulation

The paper presents design and evaluation of emotional state estimator based on artificial neural networks for physiology-driven adaptive virtual reality (VR) stimulation. Real-time emotional state estimation from physiological signals enables adapting the stimulations to the emotional response of each individual. Estimation is first evaluated on artificial subjects, which are convenient during software development and testing of physiology-driven adaptive VR stimulation. Artificial subjects are implemented in the form of parameterized skin conductance and heart rate generators that respond to emotional inputs. Emotional inputs are a temporal sequence of valence/arousal annotations, which quantitatively express emotion along unpleasant-pleasant and calm-aroused axes. Preliminary evaluation of emotional state estimation is also performed with a limited set of humans. Human physiological signals are acquired during simultaneous presentation of static pictures and sounds from valence/arousalannotated International Affective Picture System and International Affective Digitized Sounds databases.

MULTISAB project: a web platform based on specialized frameworks for heterogeneous biomedical time series analysis - an architectural overview

The aim of this work is to present an architectural overview of a novel web platform used for heterogeneous biomedical time series analysis. Its architecture is based on three subprojects: frontend, backend, and processing. Frontend uses several contemporary web technologies to present a fast, responsive and pleasing user interface. Backend, written in Java, communicates with a database and with other servers, on which the processing subproject is deployed. The processing subproject contains several frameworks intended for: record input handling, signal preprocessing, data visualization, general time series features extraction, specific time series features extraction (e.g. heart rate variability and electroencephalogram), data mining, and reporting. The platform is in an early phase of implementation, but we demonstrate its features and capabilities, of which feature extraction frameworks and signal visualization currently stand out.

A web platform for analysis of multivariate heterogeneous biomedical time-series — A preliminary report

Biomedical time-series analysis is a diverse field that includes biomedical engineering, computer science, and medical achievements, with the goal to save human lives and improve the quality of healthcare. In this paper, we present a preliminary report on construction of an innovative web platform for heterogeneous multivariate biomedical time-series analysis. The platform will feature data upload, preprocessing, feature extraction, model construction, visualization of signals and disorders, and reporting. Several scenarios of use will be prepared for a user, depending on his research or practice goals. An expert system for feature recommendation, specifically designed to support clinical decisions, will be implemented in the platform. In the current phase of research on our project, we describe the features of the platform, including some of its technological details, and the way in which we designed scenarios of use. Future papers will focus on specific aspects of the platform.

Biomedical time series preprocessing and expert-system based feature extraction in MULTISAB platform

In this paper, we review the current state of implementation of the MULTISAB platform, a web platform whose main goal is to provide a user with detailed analysis capabilities for heterogeneous biomedical time series. These time series are often encumbered by noise that prohibits accurate calculation of clinically significant features. The goal of preprocessing is either to completely remove the noise or at least to ameliorate the quality of the recorded series. The focus of this paper is on the description of an expert feature recommendation system for electrocardiogram analysis. We demonstrate the process through which one arrives at a point where significant expert features are proposed to a platform user, based on time series at hand, analysis goal, and available length of the time series. We also provide the description of implemented preprocessing techniques and feature extraction procedure within the platform.

Optimizing the detection of characteristic waves in ECG based on exploration of processing steps combinations

In this paper, algorithms for detection of characteristic waves in ECG are examined and modified. We distinguished four processing steps of detection algorithms: noise and artefacts reduction, transformations, fiducial marks selection of wave candidates, and decision rule. Several algorithms for detection of QRS, P, and T waves are explored through combinations of processing steps, in order to achieve accurate detection results. Algorithms are tested on public available ECG databases with both QRS and P and T waves annotations. We found that, depending on the database, the combination of Sun Yan’s MMF or MMD methods with Elgendi’s algorithm works best for QRS detection (Se = 99.77% +P = 99.72% for MMF on MIT-BIH Arrhythmia Database and Se = 99.90% +P = 99.89% for MMD on QT Database), while P and T waves were best detected using only Elgendi’s algorithm (P waves: Se = 60.84% +P = 59.61%, T waves: Se = 88.79% +P = 93.55% on MIT-BIH Arrhythmia Database). Our work shows that combining the best proposed methods in literature may lead to improvements in ECG waves detection, although P and T waves detection is still less than satisfactory and warrants further research.

MULTISAB: A Web Platform for Analysis of Multivariate Heterogeneous Biomedical Time-Series

There is a growing need for efficient and accurate biomedical software in healthcare community. In this paper, we present MULTISAB, a web platform whose goal is to provide users with detailed analysis capabilities for heterogeneous biomedical time series. We describe the system architecture, including its subprojects: frontend, backend and processing. Emphasis is placed on the processing subproject, implemented in Java, which incorporates data analysis methods. The subproject is divided into several frameworks: record input handling, preprocessing, signal visualization, general time series features extraction, specific (domain) time series features extraction, expert system recommendations, data mining, and reporting. Common signal features extraction framework includes a great number of features in time (both linear and nonlinear), frequency and time-frequency domain. Currently, domain specific frameworks for heart rate variability, ECG and EEG feature extraction are supported, which also include preprocessing techniques for noise reduction and detection methods for characteristic waveforms (like QRS complexes, P and T waves in ECG). Parallelization is implemented for feature extraction to increase performance. It is realized using multithreading on several levels: for multiple records, traces, and segments. Expert system is implemented, which provides automatic recommendation of the set of significant expert features that should be extracted from the analyzed signals, depending on the analysis scenario. The expert system, apart from the role in recommending features, can also participate in automatic diagnosis, after the features are extracted. Current expert system prototype contains diagnostic rules for acute myocardial ischemia, based on medical guidelines. Data mining framework contains dimensionality reduction methods and machine learning classifiers used to construct accurate and interpretable disorder models. A report is produced at the end of the process using openly available libraries. The platform includes best practices from medicine, biomedical engineering, and computer science in order to deliver detailed biomedical time series analysis services to its users.