Luciana Benotti

Engaging high school students using chatbots

Chatbots have been used in different scenarios for getting people interested in CS for decades. However, their potential for teaching basic concepts and their engaging effect has not been measured. In this paper we present a software platform called Chatbot designed to foster engagement while teaching basic CS concepts such as variables, conditionals and finite state automata, among others. We carried out two experiences using Chatbot and the well known platform Alice: 1) an online nation-wide competition, and 2) an in-class 15-lesson pilot course in 2 high schools. Data shows that retention and girl interest are higher with Chatbot than with Alice, indicating student engagement.

A Tool for Introducing Computer Science with Automatic Formative Assessment

In this paper we present a software platform called Chatbot designed to introduce high school students to Computer Science (CS) concepts in an innovative way: by programming chatbots. A chatbot is a bot that can be programmed to have a conversation with a human or robotic partner in some natural language such as English or Spanish. While programming their chatbots, students use fundamental CS constructs such as variables, conditionals, and finite state automata, among others. Chatbot uses pattern matching, state of the art lemmatization techniques, and finite state automata in order to provide automatic formative assessment to the students. When an error is found, the formative feedback generated is immediate and task-level. We evaluated Chatbot in two observational studies. An online nation-wide competition where more than 10,000 students participated. And, a mandatory in-class 15-lesson pilot course in three high schools. We measured indicators of student engagement (task completion, participation, self reported interest, etc.) and found that girls’ engagement with Chatbot was higher than boys’ for most indicators. Also, in the online competition, the task completion rate for the students that decided to use Chatbot was five times higher than for the students that chose to use the renowned animation and game programming tool Alice. Our results suggest that the availability of automatic formative assessment may have an impact on task completion and other engagement indicators among high school students.

Interpreting Natural Language Instructions Using Language, Vision, and Behavior

We define the problem of automatic instruction interpretation as follows. Given a natural language instruction, can we automatically predict what an instruction follower, such as a robot, should do in the environment to follow that instruction? Previous approaches to automatic instruction interpretation have required either extensive domain-dependent rule writing or extensive manually annotated corpora. This article presents a novel approach that leverages a large amount of unannotated, easy-to-collect data from humans interacting in a game-like environment. Our approach uses an automatic annotation phase based on artificial intelligence planning, for which two different annotation strategies are compared: one based on behavioral information and the other based on visibility information. The resulting annotations are used as training data for different automatic classifiers. This algorithm is based on the intuition that the problem of interpreting a situated instruction can be cast as a classification problem of choosing among the actions that are possible in the situation. Classification is done by combining language, vision, and behavior information. Our empirical analysis shows that machine learning classifiers achieve 77% accuracy on this task on available English corpora and 74% on similar German corpora. Finally, the inclusion of human feedback in the interpretation process is shown to boost performance to 92% for the English corpus and 90% for the German corpus.

Modeling the clarification potential of instructions

We hypothesize that implicatures are a rich source of clarification requests.We motivate the hypothesis in theoretical, practical and empirical terms.We model clarification potential by inferring conversational implicatures.Much of the inference can be handled using classical AI planning.Discourse structure emerges as task structure is exploited opportunistically. We hypothesize that conversational implicatures are a rich source of clarification requests, and in this paper we do two things. First, we motivate the hypothesis in theoretical, practical and empirical terms and formulate it as a concrete clarification potential principle: implicatures may become explicit as fourth-level clarification requests. Second, we present a framework for generating the clarification potential of an instruction by inferring its conversational implicatures with respect to a particular context. We evaluate the framework and illustrate its performance using a humanhuman corpus of situated conversations. Much of the inference required can be handled using classical planning, though as we shall note, other forms of means-ends analysis are also required. Our framework leads us to view discourse structure as emerging via opportunistic responses to task structure.

Context and Implicature

This chapter introduces Paul Grice’s notion of conversational implicature. The basic ideas—the cooperative principle, the maxims of conversation, and the contrast between implicature and presupposition—make it clear that conversational implicature is a highly contextualized form of language use that has a lot in common with non-linguistic behavior. But what exactly is its role? The authors invite the reader to view conversational implicature as a way of negotiating meaning in conversational contexts. Along the way, the reader will learn something of the theoretical properties of implicatures, why they are tricky to work with empirically, what can be done with them computationally, and (perhaps) where future research on the topic may lead. But the basic message of the chapter is actually quite simple: context and conversational implicature are highly intertwined, and unravelling their interactions is a challenging and worthwhile research goal.

The Effect of a Web-based Coding Tool with Automatic Feedback on Students' Performance and Perceptions

In this paper we do three things. First, we describe a web-based coding tool that is open-source, publicly available and provides formative feedback and assessment. Second, we compare several metrics on student performance in courses that use the tool versus courses that do not use it when learning to program in Haskell. We find that the dropout rates are significantly lower in those courses that use the tool at two different universities. Finally we apply the technology acceptance model to analyse students perceptions.

UNC++Duino: A Kit for Learning to Program Robots in Python and C++ Starting from Blocks

We present UNC++Duino, an open source educative software for learning to program a robotic kit in C++ and Python. Besides of these two industry programming languages, UNC++Duino can be programmed using 2 high level languages based on blocks are free of syntax errors. One of the block based languages included is completely iconic allowing for its use with preliterate children. The hardware we use with UNC++Duino, the open RobotGroup robotic kit, can be used to build different automated constructions based on an Arduino board, sensors and actuators. UNC++Duino was developed within Argentinean K-12 schools by the Universidad Nacional de Cordoba with the collaboration and support of the Argentinean National Ministry of Science and the RISE program in Google for Education. Its goal is to provide an engaging tool for learning to program in different programming languages with increasing difficulty and control of the hardware.

Zoom: a corpus of natural language descriptions of map locations

This paper describes an experiment to elicit referring expressions from human subjects for research in natural language generation and related fields, and preliminary results of a computational model for the generation of these expressions. Unlike existing resources of this kind, the resulting data set the Zoom corpus of natural language descriptions of map locations takes into account a domain that is significantly closer to real-world applications than what has been considered in previous work, and addresses more complex situations of reference, including contexts with different levels of detail, and instances of singular and plural reference produced by speakers of Spanish and Portuguese.

Evaluation of a Refinement Algorithm for the Generation of Referring Expressions

In this paper we describe and evaluate an algorithm for generating referring expressions that uses linear regression for learning the probability of using certain properties to describe an object in a given scene. The algorithm we present is an extension of a refinement algorithm modified to take probabilities learnt from corpora into account. As a result, the algorithm is able not only to generate correct referring expressions that uniquely identify the referents but it also generates referring expressions that are considered equal or better than those generated by humans in 92% of the cases by a human judge. We classify and give examples of the referring expressions that humans prefer, and indicate the potential impact of our work for theories of the egocentric use of language.

Content determination through planning for flexible game tutorials

The goal of this work is to design and implement an agent which generates hints for a player in a first person shooter game. The agent is a computer-controlled character which collaborates with the player to achieve the goal of the game. Such agent uses state of the art reasoning techniques from the area of artificial intelligence planning in order to come up with the content of the instructions. Moreover, it applies techniques from the area of natural language generation to generate the hints. As a result the instructions are both causally appropriate at the point in which they are uttered and relevant to the goal of the game.