Gregory Dyke

Tatiana: an environment to support the CSCL analysis process

The analysis of multimodal computer-mediated human interaction data is difficult: the diverse nature of this data and its sheer quantity is challenging enough, but a further obstacle is introduced by the complex nature of these interactions. In this paper, we describe the kinds of activities performed by researchers wishing to analyze this data. We present a model for analysis based on these activities. We then introduce Tatiana (Trace Analysis Tool for Interaction Analysts) as an environment based on this model and designed to assist researchers in managing, synchronizing, visualizing and analyzing their data by iteratively creating artifacts which further their understanding or exhibit their current understanding of their data. We show how Tatiana can be used to perform analyses and its potential for sharing corpora and analyses within the research community.

Towards an Agile Approach to Adapting Dynamic Collaboration Support to Student Needs

This paper investigates the use of conversational agents to scaffold on-line collaborative learning discussions through an approach called Academically Productive Talk (APT). In contrast to past work on dynamic support for collaborative learning, where agents were used to elevate conceptual depth by leading students through directed lines of reasoning (Kumar & Rosé, IEEE Transactions on Learning Technologies, 4(1), 2011), this APT-based approach uses generic prompts that encourage students to articulate and elaborate their own lines of reasoning, and to challenge and extend the reasoning of their teammates. This paper integrates findings from a series of studies across content domains (biology, chemistry, engineering design), grade levels (high school, undergraduate), and facilitation strategies. APT based strategies are contrasted with simply offering positive feedback when the students themselves employ APT facilitation moves in their interactions with one another, an intervention we term Positive Feedback for APT engagement. The pattern of results demonstrates that APT based support for collaborative learning can significantly increase learning, but that the effect of specific APT facilitation strategies is context specific. It appears the effectiveness of each strategy depends upon factors such as the difficulty of the material (in terms of being new conceptual material versus review) and the skill level of the learner (urban public high school vs. selective private university). In contrast, Feedback for APT engagement does not positively impact learning. In addition to an analysis based on learning gains, an automated conversation analysis technique is presented that effectively predicts which strategies are successfully operating in specific contexts. Implications for design of more agile forms of dynamic support for collaborative learning are discussed.

Towards academically productive talk supported by conversational agents

In this paper, we investigate the use of conversational agents to scaffold on-line collaborative learning discussions through an approach called academically productive talk. In contrast to past work, which has involved using agents to elevate the conceptual depth of collaborative discussion by leading students in groups through directed lines of reasoning, this approach lets students follow their own lines of reasoning and promotes productive practices such as explaining, stating agreement and disagreement, and reading and revoicing the statements of other students. We contrast two types of academically productive talk support for a discussion about 9th grade biology and show that one type in particular has a positive effect on the overall conversation, while the other is worse than no support. This positive effect carries over onto participation in a full-class discussion the following day. We use a sociolinguistic style analysis to investigate how the two types of support influence the discussion and draw conclusions for redesign. In particular, our findings have implications for how dynamic micro-scripting agents such as those scaffolding academically productive talk can be used in consort with more static macro- and micro- scripting.

Gaining Insights from Sociolinguistic Style Analysis for Redesign of Conversational Agent Based Support for Collaborative Learning

Data from an early stage of development of conversational agent based support for collaborative learning provides an ideal resource for demonstrating the value of sociolinguistic style analysis paired with time series visualizations as part of an iterative design process. The methodology illustrated in this chapter was introduced in earlier publications focusing separately on the sociolinguistic style analysis (Howley and Rose, Modeling the rhetoric of human-computer interaction. In: HCII’11 Proceedings of the 14th international conference on Human-computer interaction: interaction techniques and environments, pp 341–350, Springer-Verlag, Berlin, Heidelberg, 2011; Howley et al., A multivocal process analysis of social positioning in study groups. In: Suthers et al. (eds). Productive multivocality in the analysis of group interactions, Springer, 2013) and the time series visualization using the Tatiana tool (Dyke et al., Challenging assumptions: using sliding window visualizations to reveal time-based irregularities in CSCL processes. In: Proceedings of the international conference of the learning sciences. Sydney, Australia, 2012). However this chapter is unique in its application to data that is at such an early stage in a development process. The data is admittedly raw, and contains many examples of interaction gone awry. Nevertheless, the value in this analysis is in a demonstration of what insights can be gained through detailed stylistic analysis of conversational behavior that informs the next steps of intervention development.

Network Analytic Techniques for Online Chat

Multivocal analysis applies two or more research methods to the same data set and then applies reflexivity in a joint analysis to achieve greater insights than would be possible with a single method. In this pilot study, we demonstrate how the application of specific methods are influenced by the ordering of the methods, and present a guideline for future multivocal analysis of online chat data using network analytic techniques. We do this in two phases. First, we use Stahl’s ethnomethodological analysis of one session of biology chat discourse to inform decisions about how to identify and weight implicit connections between participants. Implicit connections are useful because they can be easily automated and presented in real time. We then contrast Stahl’s analysis with the networks we derive from those implicit connections, showing some similarities. Second, we use Tatiana to construct ethnomethodologically informed networks for the full corpora and perform network analysis on the resulting explicit connections. The results are not aligned with our first phase analysis of network position and roles for members. Further inquiry illustrates that the session chosen for ethnomethodological analysis by Stahl has different characteristics than the other six sessions, drawing our use of that analysis for building implicit connections in the corpora into question. We conclude with a clear vision for applying the Group Informatics methodological approach to corpora prior to the performance of time consuming qualitative methods like ethnomethodologically informed analysis. Weaving methods together in the right order, we argue, will lead to more rapid and deeper insight.

Group composition and intelligent dialogue tutors for impacting students' academic self-efficacy

In this paper, we explore using an intelligent dialogue tutor to influence student academic self-efficacy, as well as its interaction with group self-efficacy composition in a dyadic learning environment. We find providing additional tutor prompts encouraging students to participate in discussion may have unexpected negative effects on self-efficacy, especially on students with low self-efficacy scores who have partners with low self-efficacy scores.

Analytic Representations and Affordances for Productive Multivocality

This chapter describes and reflects upon the analytic representations used in the analyses presented in this book, and the roles they played in multivocal analysis. As shown in other chapters, multivocality across analyses based on shared datasets can be productive in a variety of ways and for a variety of reasons. From a pragmatic perspective this productivity is also dependent on the ability of analysts to share datasets, perform analyses, inscribe new analytic knowledge into representations and use these representations as a basis for discussion. In this chapter, we examine how representations are used and given meaning in analysis. We catalogue the types of entities and attributes inscribed in representations, the notational systems by which they are encoded, and the kinds of moves that result in the creation of new representations. We then discuss the opportunities for multivocality afforded by the representations present in the different data sections, and discuss the properties desirable in a framework for coordinating analytic representations. We describe instances of representation-based productive multivocality found in this volume, presenting nine strategies for researchers seeking to engage in productive multivocality. This chapter will be of interest to tool designers, but also provide guidance to researchers in reflectively choosing representations (and their affordances for interpretation and manipulation) so as to maximize their ability to engage in productive multivocality.