Theodora Chaspari

Sparse Representation of Electrodermal Activity With Knowledge-Driven Dictionaries

Biometric sensors and portable devices are being increasingly embedded into our everyday life, creating the need for robust physiological models that efficiently represent, analyze, and interpret the acquired signals. We propose a knowledge-driven method to represent electrodermal activity (EDA), a psychophysiological signal linked to stress, affect, and cognitive processing. We build EDA-specific dictionaries that accurately model both the slow varying tonic part and the signal fluctuations, called skin conductance responses (SCR), and use greedy sparse representation techniques to decompose the signal into a small number of atoms from the dictionary. Quantitative evaluation of our method considers signal reconstruction, compression rate, and information retrieval measures, that capture the ability of the model to incorporate the main signal characteristics, such as SCR occurrences. Compared to previous studies fitting a predetermined structure to the signal, results indicate that our approach provides benefits across all aforementioned criteria. This paper demonstrates the ability of appropriate dictionaries along with sparse decomposition methods to reliably represent EDA signals and provides a foundation for automatic measurement of SCR characteristics and the extraction of meaningful EDA features.

Using physiology and language cues for modeling verbal response latencies of children with ASD

Signal-derived measures can provide effective ways towards quantifying human behavior. Verbal Response Latencies (VRLs) of children with Autism Spectrum Disorders (ASD) during conversational interactions are able to convey valuable information about their cognitive and social skills. Motivated by the inherent gap between the external behavior and inner affective state of children with ASD, we study their VRLs in relation to their explicit but also implicit behavioral cues. Explicit cues include the childrens language use, while implicit cues are based on physiological signals. Using these cues, we perform classification and regression tasks to predict the duration type (short/long) and value of VRLs of children with ASD while they interacted with an Embodied Conversational Agent (ECA) and their parents. Since parents are active participants in these triadic interactions, we also take into account their linguistic and physiological behaviors. Our results suggest an association between VRLs and these externalized and internalized signal information streams, providing complementary views of the same problem.

An acoustic analysis of shared enjoyment in ECA interactions of children with autism

The quality of shared enjoyment in interactions is a key aspect related to Autism Spectrum Disorders (ASD). This paper discusses two types of enjoyment: the first refers to humorous events and is associated with ones positive affective state and the second is used to facilitate social interactions between people. These types of shared enjoyment are objectively specified by their proximity to a voiced and unvoiced laughter instance, respectively. The goal of this work is to study the acoustic differences of areas surrounding the two kinds of shared enjoyment instances, called “social zones”, using data collected from children with autism, and their parents, interacting with an Embodied Conversational Agent (ECA). A classification task was performed to predict whether a “social zone” surrounds a voiced or an unvoiced laughter instance. Our results indicate that humorous events are more easily recognized than events acting as social facilitators and that related speech patterns vary more across children compared to other interlocutors.

Pathological speech processing: State-of-the-art, current challenges, and future directions

The study of speech pathology involves evaluation and treatment of speech production related disorders affecting phonation, fluency, intonation and aeromechanical components of respiration. Recently, speech pathology has garnered special interest amongst machine learning and signal processing (ML-SP) scientists. This growth in interest is led by advances in novel data collection technology, data science, speech processing and computational modeling. These in turn have enabled scientists in better understanding both the causes and effects of pathological speech conditions. In this paper, we review the application of machine learning and signal processing techniques to speech pathology and specifically focus on three different aspects. First, we list challenges such as controlling subjectivity in pathological speech assessments and patient variability in the application of ML-SP tools to the domain. Second, we discuss feature design methods and machine learning algorithms using a combination of domain knowledge and data driven methods. Finally, we present some case studies related to analysis of pathological speech and discuss their design.

Markov Chain Monte Carlo Inference of Parametric Dictionaries for Sparse Bayesian Approximations

Parametric dictionaries can increase the ability of sparse representations to meaningfully capture and interpret the underlying signal information, such as encountered in biomedical problems. Given a mapping function from the atom parameter space to the actual atoms, we propose a sparse Bayesian framework for learning the atom parameters, because of its ability to provide full posterior estimates, take uncertainty into account and generalize on unseen data. Inference is performed with Markov Chain Monte Carlo, that uses block sampling to generate the variables of the Bayesian problem. Since the parameterization of dictionary atoms results in posteriors that cannot be analytically computed, we use a Metropolis-Hastings-within-Gibbs framework, according to which variables with closed-form posteriors are generated with the Gibbs sampler, while the remaining ones with the Metropolis Hastings from appropriate candidate-generating densities. We further show that the corresponding Markov Chain is uniformly ergodic ensuring its convergence to a stationary distribution independently of the initial state. Results on synthetic data and real biomedical signals indicate that our approach offers advantages in terms of signal reconstruction compared to previously proposed Steepest Descent and Equiangular Tight Frame methods. This paper demonstrates the ability of Bayesian learning to generate parametric dictionaries that can reliably represent the exemplar data and provides the foundation towards inferring the entire variable set of the sparse approximation problem for signal denoising, adaptation, and other applications.

Using Multimodal Wearable Technology to Detect Conflict among Couples

By monitoring human behavior unobtrusively, mobile sensing technologies have the potential to improve our daily lives. Initial results from a field study demonstrate that such passive technologies can detect a complex psychological state in an uncontrolled, real-life environment. In the web extra at https://youtu.be/n8Ap3Z44ojQ, guest editor Katarzyna Wac interviews authors Adela Timmons and Theodora Chaspari, quality-of-life technology researchers at the University of Southern California.

Exploring Children's Verbal and Acoustic Synchrony: Towards Promoting Engagement in Speech-Controlled Robot-Companion Games

Childrens interpersonal synchrony has been related to various benefits in social, mental and emotional development. We explore verbal and acoustic synchrony patterns between pairs of children playing a speech-controlled video game. Verbal features include word timing and duration patterns, while acoustic cues contain prosodic information. Synchrony is captured through a random-effects model taking into account multiple sources of variation and repeated measurements for each pair of children. Our findings indicate the presence of synchrony between participants during game play, which increases as they become more engaged in the game. These results are discussed in relation to personalized human-computer interaction and adaptive game environments.

A NON-HOMOGENEOUS POISSON PROCESS MODEL OF SKIN CONDUCTANCE RESPONSES INTEGRATED WITH OBSERVED REGULATORY BEHAVIORS FOR AUTISM INTERVENTION

Early intervention in individuals with Autism Spectrum Disorder (ASD) can improve core and associated symptoms and facilitate skills that increase social opportunities. However, determining effective intervention success in this population, and the mechanisms that produce it, is currently restricted to observable behavior. The need of therapy assessment metrics beyond traditional behavioral criteria, led to the use of physiological signals for capturing child-therapist internal dynamics during an intervention session. Internal physiological states were measured through Electrodermal Activity (EDA) and modeled in relation to observed self- and co-regulatory behaviors. A common measure of EDA, Skin Conductance Response (SCR), was the primary signal of interest and assumed to form a non-homogeneous Poisson Process whose rate function is determined by observed regulatory behaviors. Through likelihood and residual goodness of fit analysis, statistical tests and classification tasks, our results indicate that SCR changes and observable behavior in child-therapist dyads are temporally associated and the estimated model parameters can be linked to the types of regulation stimuli.

Quantifying EDA synchrony through joint sparse representation: A case-study of couples' interactions

The co-variation degree between individuals in their physiological signals can reveal insights about the quality of their interaction as well as their personal characteristics. In an effort to capture the amount of synchrony between Electrodermal Activity (EDA) streams occurring in parallel during dyadic interactions, we propose Sparse EDA Synchrony Measure (SESM), an index derived from the joint sparse representation of EDA ensembles. Sparse decomposition is performed using Simultaneous Orthogonal Matching Pursuit (SOMP) from a knowledge-driven dictionary of tonic and phasic atoms, capturing the slow-varying trends and high-frequency signal fluctuations, respectively. At each iteration the atom having the maximum average correlation with the residuals is selected. We compute SESM as the negative natural logarithm of the joint reconstruction error and evaluate it with data from interactions of married and young dating couples participating in tasks of varying emotional intensity. Through statistical analysis and multiple linear regression experiments, our results indicate that SESM depicts significant differences across tasks in both datasets considered and can be associated to individuals attachment-related characteristics.

New Frontiers in Ambulatory Assessment

With the increasing use of smartphone technologies and wearable biosensors, we are currently undergoing what many have termed a “data revolution,” where intensive, multichannel data are passively collected over long time frames. Such procedures are transforming the way psychologists conceptualize research and have the potential to spur important advances in the study of close relationships. This proof-of-concept study from the Couple Mobile Sensing Project, a partnership between psychologists and engineers, combines big data and ambulatory assessment methodologies to study multimodal, microprocesses in couples’ everyday lives. These data collection procedures are designed to test how characteristics of everyday behavioral, physiological, and vocal interactions are integrated within and across individuals. We present two mini-illustrations to show how these data can be synchronized across modalities and partners and can be linked to generalized relationship dimensions. Discussion highlights the potential and challenges of capturing multimodal, multiperson, real-time, naturally occurring social phenomena.