Sarah Ita Levitan

Entrainment, dominance and alliance in supreme court hearings

A major goal of the Cognitive Infocommunication approach is to develop applications in which human and artificial cognitive systems are made to work more effectively. A critical step in this process is improving our understanding of human–human interaction so that it may be modeled more closely. Our work addresses this task by examining the role of entrainment – the propensity of conversational partners to behave like one another – in (1) the production of conversational fillers (CFs) and acoustic intensity; (2) patterns of turn-taking; and (3) Linguistic Style. Markers and how all of these relate to power relations, conflict, and voting behavior in a corpus of speech produced by justices and lawyers during oral arguments of the U.S. Supreme Court in the 2001 term. We examine several different measures of entrainment in justice–lawyer pairs to see whether or not they are related to justices’ favorable or unfavorable votes for the lawyer’s side. While two measures (a naive measure of similarity in CF rates and global similarity in CF phonetic realizations for the entire session) show no relationship, a third, which measures local entrainment in CFs in lawyer-justice pairs, does in fact identify a significant positive relationship between entrainment and justice votes. With respect to local entrainment in intensity, we found that lawyers do entrain more to justices than justices to lawyers, although there is no greater entrainment of female lawyers than of male lawyers. When we examine the relationship between entrainment in intensity and judicial voting, we find that, when justices voted for the petitioners, there is significant evidence of entrainment by both petitioners and respondents to justices. With respect to turn-taking behavior, we find that certain patterns of overlaps in turn exchanges between justices and lawyers are correlated with justices’ voting behavior for four of the justices in our corpus. Finally, we find that there are lexical cues to divisiveness within the Court itself that can distinguish cases with close verdicts from cases with unanimous verdicts. We link these results to the possibility of building cognitive info-communication interfaces that exploit features of human–human entrainment for increasing effectiveness of human–machine interactions.

Cross-Cultural Production and Detection of Deception from Speech

Detecting deception from different dimensions of human behavior has been a major goal of research in psychology and computational linguistics for some years and is currently of considerable interest to military and law enforcement agencies. However, relatively little work has been done to develop automatic methods to detect deception from spoken language or to compare deception detection and production between different cultures. We present results of experiments on a new corpus of deceptive and non-deceptive speech, collected from native speakers of Standard American English and Mandarin Chinese, all speaking English, to investigate acoustic, prosodic, and lexical cues to deception. We report first on the role of personality factors derived from the NEO-FFI (Neuroticism-Extraversion-Openness Five Factor Inventory) and of gender, ethnicity and confidence ratings on subjects? ability to deceive and to detect deception. We then present classification results discriminating deceptive from non-deceptive speech, using these features as well as acoustic and prosodic cues. We find that combining acoustic and prosodic features with information about the speaker?s personality, gender, and language results in a classification accuracy of 65.86%, which represents ~10% relative improvement from baseline accuracy.

Combining Acoustic-Prosodic, Lexical, and Phonotactic Features for Automatic Deception Detection.

Improving methods of automatic deception detection is an important goal of many researchers from a variety of disciplines, including psychology, computational linguistics, and criminology. We present a system to automatically identify deceptive utterances using acoustic-prosodic, lexical, syntactic, and phonotactic features. We train and test our system on the Interspeech 2016 ComParE challenge corpus, and find that our combined features result in performance well above the challenge baseline on the development data. We also perform feature ranking experiments to evaluate the usefulness of each of our feature sets. Finally, we conduct a cross-corpus evaluation by training on another deception corpus and testing on the ComParE corpus.

Automatically Classifying Self-Rated Personality Scores from Speech.

Automatic personality recognition is useful for many computational applications, including recommendation systems, dating websites, and adaptive dialogue systems. There have been numerous successful approaches to classify the “Big Five” personality traits from a speaker’s utterance, but these have largely relied on judgments of personality obtained from external raters listening to the utterances in isolation. This work instead classifies personality traits based on self-reported personality tests, which are more valid and more difficult to identify. Our approach, which uses lexical and acoustic-prosodic features, yields predictions that are between 6.4% and 19.2% more accurate than chance. This approach predicts Opennessto-Experience and Neuroticism most successfully, with less accurate recognition of Extroversion. We compare the performance of classification and regression techniques, and also explore predicting personality clusters.

Identifying Individual Differences in Gender, Ethnicity, and Personality from Dialogue for Deception Detection

When automatically detecting deception, it is important to model individual differences across speakers. We explore the automatic identification of individual traits such as gender, native language, and personality, using acoustic-prosodic and lexical features from an initial non-deceptive dialogue. We also explore predicting success at deception and at deception detection, using the same features.