Juliana Jansen Ferreira

Software Developers as Users

In view of the pervasive role played by Information Technology in contemporary life, a growing community of researchers, practitioners, and educators has been involved with human-centered computing (HCC), a fi eld of studies concerned with the integration of theories and methodologies to support the combined investigation of machines, humans, and domains of applications. Comprehensive HCC studies should then articulate technical, personal, social, and cultural factors, addressing the use of technology, its design, and development. This introductory chapter provides an overview of our incremental contribution to advance HCC studies, a suite of tools called SigniFYI . With this tool, we aim to uncover meanings inscribed in software, their origins, and consequences. We propose to identify and trace instances of consistently related objects across different segments of investigation concerning software production and use. In order to achieve this goal, we rely on semiotic engineering theory, which provides us with conceptual and methodological resources with which to obtain a coherent perspective across multiple segments of investigation. The result is a principled account of relations between the objects in each segment. Following Schön’s perspectives on refl ective practice applied to software design and development, SigniFYI stimulates researchers, professionals, and educators to think critically about what they do and how they do it and with which means and for which ends. Additionally, in research contexts, SigniFYI supports the validation of knowledge produced with interpretive research methods. As cultural, political, social, psychological, and even physical life experiences of individuals all over the world become increasingly tied to the use and effects of Information Technology (IT), a growing number of researchers, practitioners, and educators are concerned with how human values affect or can be affected by software development and use. Members of this community of interest have contributed to establishing a relatively new fi eld of studies called human-centered computing (HCC) , whose aim is to integrate various disciplines that provide us with knowledge and methods with which to search answers for the questions we ask. Theoretical and methodological integration is the key to HCC. Multiple disciplines have been contributing to the study of Computer Science and Informatics, from the more abstract ones (like mathematics and logic) to the ones that are more focused on human experience (like psychology, sociology, and design). However,

Combining cognitive, semiotic and discourse analysis to explore the power of notations in visual programming

Using game design and programming to foster computational thinking acquisition has proved to be a successful strategy in recent years. In previous research with AgentSheets, we concluded that the semiotic richness of this visual programming environment, specifically designed to support computational thinking acquisition, could be explored more extensively to the benefit of learners. In particular, we realized that there are some additional representations of AgentSheets games and simulations that are not presented as programming tools in the interface, and yet they communicate new relevant meanings to the users. This paper reports on research where we artificially introduced such representations in a small follow-up experiment with selected participants from our previous research experiment. Our goal was to investigate the impact of such additional representations on program comprehension and modification tasks. To this end we contrasted empirical evidence of their performance in the two tasks with their verbal account of experience with AgentSheets. We used a combination of discourse analysis and inspections using Semiotic Engineering methods and the Cognitive Dimensions of Notations framework. Our findings go in two directions. First, we observed that additional representations have allowed participants to expand and correct previous learning. Therefore such representations can support new teaching strategies in computational thinking acquisition programs with AgentSheets. Second, we learned that the combination of methods we used to analyze empirical data - discourse analysis with semiotic and cognitive inspection techniques - can be used systematically in other research contexts, holding the promise of insightful results.

On signifying the complexity of inter-agent relations in agentsheets games and simulations

This paper reports the results of an empirical study about the semiotic engineering of signs of complexity for live documentation of games and simulations built with a visual programming learning environment. The study highlights the essence of the semiotic engineering process and shows how its outcome has been received by a group of users who can speak for a large portion of the live documentation systems user population. It also shows how the communication of complexity is, in and of itself, a major design challenge, especially when mastering complexity is one of the prime purposes of the documented object. Because the study was carried out in the context of a live documentation system, its conclusions can also illustrate how to conduct semiotically-inspired interaction design.

Abstraction Levels as Support for UX Design of User’s Interaction Logs

User interaction logging is a powerful tool for user behavior studies, usability testing, and system metrics analysis. It may also be applied in large data contexts, such as social networks analysis, helping data scientists to understand social patterns. Data scientists, User Experience (UX) designers, Human-Computer Interaction (HCI) practitioners, and software engineers have been performing the analysis of this kind of data to obtain knowledge regarding the source system’s usage. User interaction log data, however, can also be critical for final users themselves. They can use interaction log data, for example, (i) to revisit his own interaction path, redoing his steps that lead to a relevant insight or discovery; (ii) to learn from someone else’s interaction path new ways to perform a given task; (iii) or even to analyze critical steps of a process supported by the source system. The need for final users to consume interaction log data is presenting significant challenges for UX researchers. Influenced by Semiotic Engineering, a HCI theory that views human-computer interaction as a form of human communication between designers and users mediated by a computer system, we propose three user interaction log abstraction levels - strategic, tactical, and operational - to frame and guide user interaction logs’ UX design. In this paper, we discuss how those abstraction levels can be used as UX design guidelines and present some research questions to be explored - how source system captures interaction log is central for log analysis strategy and how a strategic level can be identified thought the analysis of interaction logs data from other abstraction levels.

Communicability Issues on PaaS Application Development

Platform as a Service (PaaS) is becoming a differential product for big technology companies. It delivers hardware, software tools and other resources for application development and hosting, as a service. Its users are developers who need to build and deploy new applications. Besides computational power, PaaS environments (PaaSE) offer services, development tools or even complete apps to be putted together in final applications. These pieces of software can be developed by different parties, presenting a significant challenge from the HCI perspective. Semiotic Engineering (SemEng) views HCI as computer-mediated communication between designers and users at interaction time. In the PaaS context, several designers communicate with PaaSEs users (developers). In this paper, we apply SemEng concepts to analyze diverse software artifacts involved, showing evidence of communication breakdowns between designers and users. Our goal is to provide a better understanding of existing metacommunication processes in PaaSE, offering specific suggestions to emphasize communication boundaries.

Cognitive dimensions of notation tailored to environments for visualization and insights

Visualization diversity is a core resource for interpreting and getting insights about data. Different representations can support real-time decision-making and make more knowledge available for forecast and planning processes involving historical data analysis. Cognitive characteristics of visualization resources are crucial for visualization and insights environments. We present an initial tailored definition of Cognitive Dimensions of Notations (CDN) to support the evaluation and (re)design of such an environment. Using WISE (Weather InSights Environment) as a use case, we show evidences of how the tailored CDN can support HCI evaluation and discussions about the (re)design.

CoRgI: Cognitive Reasoning Interface

Cognitive systems built with artificial intelligence resources (AI-powered systems) can be defined as software systems that learn at scale from their interaction with humans and environments. That kind of system allows people to augment their cognitive potential in order to harvest insights from huge quantities of data to understand complex situations, make accurate predictions about the future, and anticipate the unintended consequences of actions. These systems evolve naturally from such learning, rather than being explicitly programmed. In this approach, humans and computers work more interconnected to achieve unexpected insights. In order to be useful, an AI-powered system must be aware of the users’ goals, so it can help him/her by bringing contextual information from multiple sources, guiding through the series of tasks associated with the goals. The knowledge structuring is a challenge by itself and it has been the focus of knowledge engineering research. Once the knowledge is structured or organized, the challenge falls on UX researchers to investigate users and their tasks and goals with that structured knowledge. Questions like who the users are, what they want or need to do, in which preferred ways, and what are users’ goals can guide UX research on this matter. We argue that an AI-powered system could infer the user’s goals by observing his/her interactions with different systems and considering its knowledge base – about the user, the group(s) s/he is part of, the applications’ domains, the overall context, etc. Based on fieldwork executed for a project where a knowledge-intensive process is analyzed and discussed with support of an AI-Powered System, we propose the Cognitive Reasoning Interface (CoRgI) framework. This paper presents the fieldwork observation that led to the development of the framework, how we conceptualize the framework, and our initial validation of the framework.

An eye gaze model for seismic interpretation support

Designing systems to offer support to experts during cognitive intensive tasks at the right time is still a challenging endeavor, despite years of research progress in the area. This paper proposes a gaze model based on eye tracking empirical data to identify when a system should proactively interact with the expert during visual inspection tasks. The gaze model derives from the analyses of a user study where 11 seismic interpreters were asked to perform the visual inspection task of seismic images from known and unknown basins. The eye tracking fixation patterns were triangulated with pupil dilations and thinking-aloud data. Results show that cumulative saccadic distances allow identifying when additional information could be offered to support seismic interpreters, changing the visual search behavior from exploratory to goal-directed.

AGENTES NO AGENTSHEETS: COMO O AGENTSHEETS COMUNICA O CONCEITO DE AGENTES

This paper presents findings and observations on an interesting study of AgentSheets® with a focus on the communication of the concept of agent by the designers of the tool. The concept of agent is a key concept of AgentSheets®, even the tool name carries the related term. This study goal was to investigate how the concept is communicated to the users for the tool use. The study was composed by a Semiotic Inspection Method (SIM), triangulated with assisted user interaction. The interesting points and comments presented some promising indications for future investigations. The paper presets the research held during a graduation course of Introduction to HCI (Human computer interaction) of Informatics department of PUC-RIO.

Multimedia in Cognitive-Intensive Practices: A Case with ATLAS.ti Supporting HCI Qualitative Research

Human-Computer Interaction (HCI) area uses qualitative research methods to guide its user experience studies. These methods involve collecting and analyzing large amounts of empirical materials registered in different multimedia contents. Qualitative Data Analysis (QDA) tools support HCI qualitative research activities in some level, but the cognitive-intense work of producing structured and related data, building meaningful knowledge from it, and making decisions based on this knowledge is mainly the HCI researchers duty. We present a case with Atlas.ti 7, a QDA tool, and in that scenario, we identify and assess key features that support HCI qualitative research. We discuss how those features could evolve to augment human capacity and understanding throughout the qualitative data analysis, considering the cognitive computing approach. We considered multimedia concepts to discuss the evolution of QDA tools as potential investigation paths for research in multimedia and cognitive systems.