Nikolaos Malandrakis

Multimodal Prediction of Affective Dimensions and Depression in Human-Computer Interactions

Depression is one of the most common mood disorders. Technology has the potential to assist in screening and treating people with depression by robustly modeling and tracking the complex behavioral cues associated with the disorder (e.g., speech, language, facial expressions, head movement, body language). Similarly, robust affect recognition is another challenge which stands to benefit from modeling such cues. The Audio/Visual Emotion Challenge (AVEC) aims toward understanding the two phenomena and modeling their correlation with observable cues across several modalities. In this paper, we use multimodal signal processing methodologies to address the two problems using data from human-computer interactions. We develop separate systems for predicting depression levels and affective dimensions, experimenting with several methods for combining the multimodal information. The proposed depression prediction system uses a feature selection approach based on audio, visual, and linguistic cues to predict depression scores for each session. Similarly, we use multiple systems trained on audio and visual cues to predict the affective dimensions in continuous-time. Our affect recognition system accounts for context during the frame-wise inference and performs a linear fusion of outcomes from the audio-visual systems. For both problems, our proposed systems outperform the video-feature based baseline systems. As part of this work, we analyze the role played by each modality in predicting the target variable and provide analytical insights.

SAIL: Sentiment Analysis using Semantic Similarity and Contrast Features

This paper describes our submission to SemEval2014 Task 9: Sentiment Analysis in Twitter. Our model is primarily a lexicon based one, augmented by some preprocessing, including detection of MultiWord Expressions, negation propagation and hashtag expansion and by the use of pairwise semantic similarity at the tweet level. Feature extraction is repeated for sub-strings and contrasting sub-string features are used to better capture complex phenomena like sarcasm. The resulting supervised system, using a Naive Bayes model, achieved high performance in classifying entire tweets, ranking 7th on the main set and 2nd when applied to sarcastic tweets.

Tweester at SemEval-2016 Task 4: Sentiment Analysis in Twitter using Semantic-Affective Model Adaptation

We describe our submission to SemEval2016 Task 4: Sentiment Analysis in Twitter. The proposed system ranked first for the subtask B. Our system comprises of multiple independent models such as neural networks, semantic-affective models and topic modeling that are combined in a probabilistic way. The novelty of the system is the employment of a topic modeling approach in order to adapt the semantic-affective space for each tweet. In addition, significant enhancements were made in the main system dealing with the data preprocessing and feature extraction including the employment of word embeddings. Each model is used to predict a tweet’s sentiment (positive, negative or neutral) and a late fusion scheme is adopted for the final decision.

A quantitative analysis of gender differences in movies using psycholinguistic normatives

Direct content analysis reveals important details about movies including those of gender representations and potential biases. We investigate the differences between male and female character depictions in movies, based on patterns of language used. Specifically, we use an automatically generated lexicon of linguistic norms characterizing gender ladenness. We use multivariate analysis to investigate gender depictions and correlate them with elements of movie production. The proposed metric differentiates between male and female utterances and exhibits some interesting interactions with movie genres and the screenplay writer gender.

AFFECTIVE LANGUAGE MODEL ADAPTATION VIA CORPUS SELECTION

Motivated by methods used in language modeling and grammar induction, we propose the use of pragmatic constraints and perplexity as criteria to filter the unlabeled data used to generate the semantic similarity model. We investigate unsupervised adaptation algorithms of the semantic-affective models proposed in [1, 2]. Affective ratings at the utterance level are generated based on an emotional lexicon, which in turn is created using a semantic (similarity) model estimated over raw, unlabeled text. The proposed adaptation method creates task-dependent semantic similarity models and task-dependent word/term affective ratings. The proposed adaptation algorithms are tested on anger/distress detection of transcribed speech data and sentiment analysis in tweets showing significant relative classification error reduction of up to 10%.

GOAALLL!: Using sentiment in the world cup to explore theories of emotion ☆

Abstract Sporting events evoke strong emotions among fans and thus act as natural laboratories to explore emotions and how they unfold in the wild. Computational tools, such as sentiment analysis, provide new ways to examine such dynamic emotional processes. In this article we use sentiment analysis to examine tweets posted during 2014 World Cup. Such analysis gives insight into how people respond to highly emotional events, and how these emotions are shaped by contextual factors, such as prior expectations, and how these emotions change as events unfold over time. Here we report on some preliminary analysis of a World Cup twitter corpus using sentiment analysis techniques. After performing initial tests of validation for sentiment analysis on data in this corpus, we show these tools can give new insights into existing theories of what makes a sporting match exciting. This analysis seems to suggest that, contrary to assumptions in sports economics, excitement relates to expressions of negative emotion. The results are discussed in terms of innovations in methodology and understanding the role of emotion for “tuning in” to real world events. We also discuss some challenges that such data present for existing sentiment analysis techniques and discuss future analysis.

Objective Language Feature Analysis in Children with Neurodevelopmental Disorders During Autism Assessment.

Lexical planning is an important part of communication and is reflective of a speaker’s internal state that includes aspects of affect, mood, as well as mental health. Within the study of developmental disorders such as autism spectrum disorder (ASD), language acquisition and language use have been studied to assess disorder severity and expressive capability as well as to support diagnosis. In this paper, we perform a language analysis of children focusing on word usage, social and cognitive linguistic word counts, and a few recently proposed psycholinguistic norms. We use data from conversational samples of verbally fluent children obtained during Autism Diagnostic Observation Schedule (ADOS) sessions. We extract the aforementioned lexical cues from transcripts of session recordings and demonstrate their role in differentiating children diagnosed with Autism Spectrum Disorder from the rest. Further, we perform a correlation analysis between the lexical norms and ASD symptom severity. The analysis reveals an increased affinity by the interlocutor towards use of words with greater feminine association and negative valence.

Linguistic analysis of differences in portrayal of movie characters

A computer implemented method for analyzing media content includes a step of providing a plurality of narrative files formatted in human readable format. Each narrative file includes a script and/or dialogues tagged with character names along with auxiliary information. Each script includes a plurality of portrayals performed by an associated actor or character. Linguistic representations of content of the narrative files in both abstract and semantic forms is determined. The linguistic representations are connected to higher order representations and mental states. The linguistic representations are connected to behavior and action. Interplay between language constructs and demographics of content creators is analyzed. Content representations towards individuals/groups are adapted to reflect heterogeneity in preferences.

The ELISA Situation Frame extraction for low resource languages pipeline for LoReHLT’2016

This paper describes the Situation Frame extraction pipeline developed by team ELISA as a part of the DARPA Low Resource Languages for Emergent Incidents program. Situation Frames are structures describing humanitarian needs, including the type of need and the location affected by it. Situation Frames need to be extracted from text or speech audio in a low resource scenario where little data, including no annotated data, are available for the target language. Our Situation Frame pipeline is the final step of the overall ELISA processing pipeline and accepts as inputs the outputs of the ELISA machine translation and named entity recognition components. The inputs are processed by a combination of neural networks to detect the types of needs mentioned in each document and a second post-processing step connects needs to locations. The resulting Situation Frame system was used during the first yearly evaluation on extracting Situation Frames from text, producing encouraging results and was later successfully adapted to the speech audio version of the same task.

GOAALLL!: Using sentiment in the World Cup to explore theories of emotion

Sporting events evoke strong emotions amongst fans and thus act as natural laboratories to explore emotions and how they unfold in the wild. Computational tools, such as sentiment analysis, provide new ways to examine such dynamic emotional processes. In this article we use sentiment analysis to examine tweets posted during 2014 World Cup. Such analysis gives insight into how people respond to highly emotional events, and how these emotions are shaped by contextual factors, such as prior expectations, and how these emotions change as events unfold over time. Here we report on some preliminary analysis of a World Cup twitter corpus using sentiment analysis techniques. We show these tools can give new insights into existing theories of what makes a sporting match exciting. This analysis seems to suggest that, contrary to assumptions in sports economics, excitement relates to expressions of negative emotion. We also discuss some challenges that such data present for existing sentiment analysis techniques and discuss future analysis.