Heloisa Candello

Talking with Conversational Agents in Collaborative Action

This one-day workshop intends to bring together both academics and industry practitioners to explore collaborative challenges in speech interaction. Recent improvements in speech recognition and computing power has led to conversational interfaces being introduced to many of the devices we use every day, such as smartphones, watches, and even televisions. These interfaces allow us to get things done, often by just speaking commands, relying on a reasonably well understood single-user model. While research on speech recognition is well established, the social implications of these interfaces remain underexplored, such as how we socialise, work, and play around such technologies, and how these might be better designed to support collaborative collocated talk-in-action. Moreover, the advent of new products such as the Amazon Echo and Google Home, which are positioned as supporting multi-user interaction in collocated environments such as the home, makes exploring the social and collaborative challenges around these products, a timely topic. In the workshop, we will review current practices and reflect upon prior work on studying talk-in-action and collocated interaction. We wish to begin a dialogue that takes on the renewed interest in research on spoken interaction with devices, grounded in the existing practices of the CSCW community.

Shaping the Experience of a Cognitive Investment Adviser

In this paper we describe the design process of a multi-bot conversational system to assist people to make more informed decisions about finance. Several user activities were held to understand the experience of investment decisions, the opportunities to design financial cognitive advisers, and the user perceptions of such systems. Valuable information was gathered from four user studies which assisted the project team to decide what would be the best approach to help people to make more informed decisions about investments using technology. The user studies findings highlighted that financial decisions are made based on information people receive from friends, news, and social networks, which led us to explore intelligent systems that would gather such information and play the role of financial advisers in a multiparty conversational system. We discuss the main design implications of our studies in the context of a prototype called CognIA and conclude discussing several challenges of designing conversational systems.

Building a Question-Answering Corpus Using Social Media and News Articles

Is it possible to develop a reliable QA-Corpus using social media data? What are the challenges faced when attempting such a task? In this paper, we discuss these questions and present our findings when developing a QA-Corpus on the topic of Brazilian finance. In order to populate our corpus, we relied on opinions from experts on Brazilian finance that are active on the Twitter application. From these experts, we extracted information from news websites that are used as answers in the corpus. Moreover, to effectively provide rankings of answers to questions, we employ novel word vector based similarity measures between short sentences (that accounts for both questions and Tweets). We validated our methods on a recently released dataset of similarity between short Portuguese sentences. Finally, we also discuss the effectiveness of our approach when used to rank answers to questions from real users.

Bottester: Testing Conversational Systems with Simulated Users

Recently, conversation agents have attracted the attention of many companies such as IBM, Facebook, Google, and Amazon which have focused on developing tools or API (Application Programming Interfaces) for developers to create their own chat-bots. In this paper, we focus on new approaches to evaluate such systems presenting some recommendations resulted from evaluating a real chatbot use case. Testing conversational agents or chatbots is not a trivial task due to the multitude aspects/tasks (e.g., natural language understanding, dialog management and, response generation) which must be considered separately and as a mixture. Also, the creation of a general testing tool is a challenge since evaluation is very sensitive to the application context. Finally, exhaustive testing can be a tedious task for the project team what creates a need for a tool to perform it automatically. This paper opens a discussion about how conversational systems testing tools are essential to ensure well-functioning of such systems as well as to help interface designers guiding them to develop consistent conversational interfaces.

Cognitive Computing: Where Big Data Is Driving Us

In this chapter we will discuss the concepts and challenges to design Cognitive Systems. Cognitive Computing is the use of computational learning systems to augment cognitive capabilities in solving real world problems. Cognitive systems are designed to draw inferences from data and pursue the objectives they were given. The era of big data is the basis for innovative cognitive solutions that cannot rely on traditional systems. While traditional computers must be programmed by humans to perform specific tasks, cognitive systems will learn from their interactions with data and humans. Not only is Cognitive Computing a fundamentally new computing paradigm for tackling real world problems, exploiting enormous amounts of data using massively parallel machines, but also it engenders a new form of interaction between humans and computers. As machines start to enhance human cognition and help people make better decisions, new issues arise for research. We will address these questions for Cognitive Systems: What are the needs? Where to apply? Which are the sources of information to relying on?

User Methods and Approaches to Design Cognitive Systems

This paper presents the results of a review of existing literature on user research practices for designing cognitive systems. Three databases were analyzed to review the user methods and approaches researchers apply in this field. It was considered methods and approaches aimed to gather user information and provide insights to design systems that augment human knowledge. As a result 82 papers were examined. It was clear the design process of Cognitive systems depends of user input and interaction to be successful; therefore new research methods are necessary to investigate how design artifacts might influence in decision-making, considering user interpretation, trust and confidence.

Understanding Fiado : Informal Credit in Brazil

This paper will explore insights gathered from fieldwork activities in the Northeast Brazil to design a financial application to facilitate access to credit. We observed the everyday financial practices of small merchants, their social physical networks and mechanisms to promote trust in those communities. We observed a particular kind of transaction that was common amongst our participants - called Fiado. Fiado transactions are a store credit practice in which merchants sell products to a customer based on trust that the customer will pay for it in the future. The Financial app aims to assist small merchants to manage their Fiado and their business.

A Validation Study of a Visual Analytics Tool with End Users

In this paper we describe an user evaluation that aimed to understand how a group of endusers interpret a visual analytics tool in the context of service delivery. It is common for service factories to have an organization devoted to handle incidents. Many incident management systems have strict controls on how fast incidents should be handled, often subjected to penalties when targets are not met.We call Time-Bounded Incident Management (TBIM) those systems, which require clearly defined incident resolution times. In our project, research scientists proposed a method and a visual representation named Workload Profile Chart (WPC) that had as primary goal to understand the area of incident management in a service delivery department. The objective of this visual representation is to help characterizing the performance of TBIM systems and diagnosing major issues such as resource and skill allocation problems, abnormal behavior, and incident characteristics. Researchers wanted to understand if end-users, the quality analysts (QAs), would comprehend the charts and would be able to use them to identify problems and propose effective improvement actions related to TBIM activities. The study was conducted with ten QAs of a service delivery department of a IT company based in Brazil. The data was analyzed using descriptive statistical and qualitative methods. As a result, participants were mainly guided by the axes titles and chart legends to interpret the visualizations, and not always understood what kind of data the chart was displaying. Those results served as insights of how QAs think when analyzing TBIM information in a service delivery department and what improvements in the visual representation tool may be proposed to facilitate their activity. At last we identified evidences of how to design better visual analytics tools based on participant’s perceptions and interpretations of color differences and verbal information in chart labels and legend.

The Role of Dialogue User Data in the Information Interaction Design of Conversational Systems

Designers face several challenges when designing information for conversational systems. In this paper, we discuss those challenges in the context of Information Interaction Design and the role of conversational data to address them. Using an actual study performed prior to the development of a financial chatbots adviser, we identified a set of common issues which led to 18 design recommendations. We categorize those recommendations according to the disciplines they are related (Information Design, Interaction Design, and Sensorial Design). The guidelines were employed by the actual developers of the system, simplifying considerably the development, and show the importance of actual user conversational data in the design process of conversational systems.

Recovering from Dialogue Failures Using Multiple Agents in Wealth Management Advice

In this paper, we discuss dialogue failures and how this affects the user experience, specifically for a scenario of a multi-bot conversational system. Additionally, we show how the use of multiple chatbots provide new strategies to overcome misunderstandings and to keep the user in the conversation flow. To inform such conclusions and recommendations, we describe a study with a multi-bot wealth management advice system in which participants conversed with four chatbots simultaneously. We analyzed each conversation log applying thematic network analysis and manually identified the main instances of dialogue failures, usually provoked by chatbot misunderstandings or system breakdowns. We examined the follow-up users’ utterances after each failure, and the users’ strategies to deal with them. We categorize our findings into a list of the most common users’ strategies, and highlight solutions provided by a multi-bot approach in assisting the dialogue failures.