Elijah Mayfield

The ACODEA framework: Developing segmentation and classification schemes for fully automatic analysis of online discussions

Research related to online discussions frequently faces the problem of analyzing huge corpora. Natural Language Processing (NLP) technologies may allow automating this analysis. However, the state-of-the-art in machine learning and text mining approaches yields models that do not transfer well between corpora related to different topics. Also, segmenting is a necessary step, but frequently, trained models are very sensitive to the particulars of the segmentation that was used when the model was trained. Therefore, in prior published research on text classification in a CSCL context, the data was segmented by hand. We discuss work towards overcoming these challenges. We present a framework for developing coding schemes optimized for automatic segmentation and context-independent coding that builds on this segmentation. The key idea is to extract the semantic and syntactic features of each single word by using the techniques of part-of-speech tagging and named-entity recognition before the raw data can be segmented and classified. Our results show that the coding on the micro-argumentation dimension can be fully automated. Finally, we discuss how fully automated analysis can enable context-sensitive support for collaborative learning.

Oh dear stacy!: social interaction, elaboration, and learning with teachable agents

Understanding how children perceive and interact with teachable agents (systems where children learn through teaching a synthetic character embedded in an intelligent tutoring system) can provide insight into the effects of so-cial interaction on learning with intelligent tutoring systems. We describe results from a think-aloud study where children were instructed to narrate their experience teaching Stacy, an agent who can learn to solve linear equations with the students help. We found treating her as a partner, primarily through aligning oneself with Stacy using pronouns like you or we rather than she or it significantly correlates with student learning, as do playful face-threatening comments such as teasing, while elaborate explanations of Stacys behavior in the third-person and formal tutoring statements reduce learning gains. Additionally, we found that the agents mistakes were a significant predictor for students shifting away from alignment with the agent.

Using feature construction to avoid large feature spaces in text classification

Feature space design is a critical part of machine learning. This is an especially difficult challenge in the field of text classification, where an arbitrary number of features of varying complexity can be extracted from documents as a preprocessing step. A challenge for researchers has consistently been to balance expressiveness of features with the size of the corresponding feature space, due to issues with data sparsity that arise as feature spaces grow larger. Drawing on past successes utilizing genetic programming in similar problems outside of text classification, we propose and implement a technique for constructing complex features from simpler features, and adding these more complex features into a combined feature space which can then be utilized by more sophisticated machine learning classifiers. Applying this technique to a sentiment analysis problem, we show encouraging improvement in classification accuracy, with a small and constant increase in feature space size. We also show that the features we generate carry far more predictive power than any of the simple features they contain.

Gaining Insights from Sociolinguistic Style Analysis for Redesign of Conversational Agent Based Support for Collaborative Learning

Data from an early stage of development of conversational agent based support for collaborative learning provides an ideal resource for demonstrating the value of sociolinguistic style analysis paired with time series visualizations as part of an iterative design process. The methodology illustrated in this chapter was introduced in earlier publications focusing separately on the sociolinguistic style analysis (Howley and Rose, Modeling the rhetoric of human-computer interaction. In: HCII’11 Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments, pp 341–350, Springer-Verlag, Berlin, Heidelberg, 2011; Howley et al., A multivocal process analysis of social positioning in study groups. In: Suthers et al. (eds). Productive multivocality in the analysis of group interactions, Springer, 2013) and the time series visualization using the Tatiana tool (Dyke et al., Challenging assumptions: using sliding window visualizations to reveal time-based irregularities in CSCL processes. In: Proceedings of the international conference of the learning sciences. Sydney, Australia, 2012). However this chapter is unique in its application to data that is at such an early stage in a development process. The data is admittedly raw, and contains many examples of interaction gone awry. Nevertheless, the value in this analysis is in a demonstration of what insights can be gained through detailed stylistic analysis of conversational behavior that informs the next steps of intervention development.

A Multivocal Process Analysis of Social Positioning in Study Groups

This chapter compares two multidimensional analyses of the PLTL Chemistry dataset, which each include a cognitive, relational, and motivational dimension. These multidimensional analyses serve to highlight the ways in which the complementary perspectives on collaborative processes offered by each dimension can be integrated in a way that offers deep insights into social positioning within collaborative groups. Differences revealed particularly along the relational and motivational dimensions raise important questions regarding the operationalization of interaction style as displayed through language and highlight the value of multivocality for the purpose of refining important constructs in ways that work towards theory building through integration of findings across research groups that employ different analytic frameworks coming from a common theoretical foundation.

Automating annotation of information-giving for analysis of clinical conversation

OBJECTIVE Coding of clinical communication for fine-grained features such as speech acts has produced a substantial literature. However, annotation by humans is laborious and expensive, limiting application of these methods. We aimed to show that through machine learning, computers could code certain categories of speech acts with sufficient reliability to make useful distinctions among clinical encounters. MATERIALS AND METHODS The data were transcripts of 415 routine outpatient visits of HIV patients which had previously been coded for speech acts using the Generalized Medical Interaction Analysis System (GMIAS); 50 had also been coded for larger scale features using the Comprehensive Analysis of the Structure of Encounters System (CASES). We aggregated selected speech acts into information-giving and requesting, then trained the machine to automatically annotate using logistic regression classification. We evaluated reliability by per-speech act accuracy. We used multiple regression to predict patient reports of communication quality from post-visit surveys using the patient and provider information-giving to information-requesting ratio (briefly, information-giving ratio) and patient gender. RESULTS Automated coding produces moderate reliability with human coding (accuracy 71.2%, κ=0.57), with high correlation between machine and human prediction of the information-giving ratio (r=0.96). The regression significantly predicted four of five patient-reported measures of communication quality (r=0.263-0.344). DISCUSSION The information-giving ratio is a useful and intuitive measure for predicting patient perception of provider-patient communication quality. These predictions can be made with automated annotation, which is a practical option for studying large collections of clinical encounters with objectivity, consistency, and low cost, providing greater opportunity for training and reflection for care providers.

Group composition and intelligent dialogue tutors for impacting students' academic self-efficacy

In this paper, we explore using an intelligent dialogue tutor to influence student academic self-efficacy, as well as its interaction with group self-efficacy composition in a dyadic learning environment. We find providing additional tutor prompts encouraging students to participate in discussion may have unexpected negative effects on self-efficacy, especially on students with low self-efficacy scores who have partners with low self-efficacy scores.

An analysis of perspectives in interactive settings

In this paper we investigate the effect of the context of interaction on the extent to which a contributors perspective bias is displayed through their lexical choice. We present a series of experiments on political discussion data. Our experiments indicate that (i) when people quote contributors with an opposing view, they tend to quote the words that are less strongly associated with the opposing view. (ii) Nevertheless, in quoting their opponents, the displayed bias of their word distributions shifts towards that of their opponents. (iii) The personal bias of the speaker is displayed most clearly through the words that are not quoted, (iv) although characteristics of the quoted message do have a measurable effect on the words that are included in the contribution. And, finally, (v) posts are influenced by the displayed bias of previous posts in a thread.

Discovering habits of effective online support group chatrooms

For users of online support groups, prior research has suggested that a positive social environment is a key enabler of coping. Typically, demonstrating such claims about social interaction would be approached through the lens of sentiment analysis. In this work, we argue instead for a multifaceted view of emotional state, which incorporates both a static view of emotion (sentiment) with a dynamic view based on the behaviors present in a text. We codify this dynamic view through data annotations marking information sharing, sentiment, and coping efficacy. Through machine learning analysis of these annotations, we demonstrate that while sentiment predicts a users stress at the beginning of a chat, dynamic views of efficacy are stronger indicators of stress reduction.

Computational representation of discourse practices across populations in task-based dialogue

In this work, we employ quantitative methods to describe the discourse practices observed in a direction giving task. We place a special emphasis on comparing differences in strategies between two separate populations and between successful and unsuccessful groups. We isolate differences in these strategies through several novel representations of discourse practices. We find that information sharing, instruction giving, and social feedback strategies are distinct between subpopulations in empirically identifiable ways.