Chih-Pin Hsiao

An Enactive Model of Creativity for Computational Collaboration and Co-creation

The modern landscape of computing has rapidly evolved with breakthroughs in new input modalities and interaction designs, but the fundamental model of humans giving commands to computers is still largely dominant. A small but growing number of projects in the computational creativity field are beginning to study and build creative computers that are able to collaborate with human users as partners by simulating, to various degrees, the collaboration that naturally occurs between humans in creative domains (Biles, Leonardo, 36:43–45, 2003; Lubart, Int J Hum Comput Stud, 63:365–369, 2005; Hoffman and Weinberg, Shimon: an interactive improvisational robotic marimba player. In: CHI’10 extended abstracts on human factors in computing systems, ACM, New York, pp 3097–3102, 2010; Zook et al., Understanding human creativity for computational play. In: 8th ACM conference on creativity and cognition, 2011; Davis et al., Building artistic computer colleagues with an enactive model of creativity, 2014). If this endeavor proves successful, the implications for HCI and the field of computing in general could be significant. Creative computers could understand and work alongside humans in a new hybrid form of human-computer co-creativity that could inspire, motivate, and perhaps even teach creativity to human users through collaboration.

Drawing Apprentice: An Enactive Co-Creative Agent for Artistic Collaboration

This paper describes a co-creative web-based drawing application called the Drawing Apprentice. This system collaborates with users in real time abstract drawing. We describe the theory, interaction design, and user experience of the Drawing Apprentice system. We evaluate the system with formative user studies and expert evaluations from a juried art competition in which a Drawing Apprentice submission won the code-based art category.

Tangible games for stroke rehabilitation with digital box and blocks test

The Digital Box and Block Test (DBBT) is a post-stroke rehabilitation assessment apparatus for in-home use. It is based on the clinically validated Box and Block Test used to measure unilateral gross manual dexterity. We show that the automated DBBT scoring algorithms achieve at least 90% accuracy on a scoring procedure that is traditionally labor intensive and subjective. Furthermore, we propose a tangible gaming system based on DBBT to increase patient motivation and make rehabilitation exercises more enjoyable.

Home-based computerized cognitive assessment tool for dementia screening

With a rapidly aging population worldwide, developing alternatives to enhance current cognitive screening practices is becoming increasingly important. The Clock Drawing Test, a paper-and-pencil test, has been used as one of the most popular cognitive screening tools for dementia. In this paper, we present our approach to developing a home-based computerized dementia screening tool, the ClockMe System, which we developed based on our observational study of the current practice of dementia screening at a clinic. The ClockMe System has two main parts: The ClockReader Application and the ClockAnalyzer Application. By using the ClockReader Application, older adults can self-administer dementia screening at home. The ClockAnalyzer Application enables medical practitioners to review and monitor the screening results of their patients. We conclude our paper with preliminary user evaluation results and suggestions for future implementation. The study shows the potential of computing technologies that can advance the current practice of dementia screening.

An Enactive Characterization of Pretend Play

This paper presents the result of an empirical study of 32 adult dyads (i.e. groups of two people) engaged in pretend play. Our analysis indicates that participatory sense-making plays a key role in the success of pretend play sessions. We use the cognitive science theory of enaction as a theoretical lens to analyze the empirical data given its robust conceptual framework for describing participatory sense-making. We present here five enactive characteristics of pretend play that appear to be necessary and sufficient for the emergence and maintenance of successful pretend play -- mental preparation, meaning building, narrative enaction, narrative deepening, and flow maintenance. This enactive formalization is used to propose a computational model of pretend play that can be used to design an agent capable of playing in real time with human users.

Tactile Teacher: Sensing Finger Tapping in Piano Playing

In a piano lesson, a student often imitates the teacher--s playing in terms of speed, dynamics, and fingering. While this learning model leverages ones visual and even audial perception for emulation, it still lacks an important component of piano playing -- the tactile sensation. We seek to convey the tactile sensations of the teachers keystrokes and then signal the students corresponding fingers. We implemented an instrumented fingerless glove called Tactile Teacher to detect finger taps on hard surfaces. Since finger taps generate acoustic signals and cause vibrations, we embedded three vibration sensors on the glove and use machine learning algorithms to analyze the data from the sensors. After a brief training procedure, this prototype can accurately identify single finger tap in a very good performance at above 89% accuracy, and two finger taps resulted in accuracy around 85%.

Sketch master: a sketch game for collecting exploratory data

We present the concept and implementation of a sketching game called Sketch Master. Sketch Master is a game designed to help players learn and practice drawing from memory. The architecture of the tool and its various game modes are presented. Additionally, we describe how functions in Sketch Master serve as a research instrument to collect exploratory data about the relation between perception, memory, and sketching.

Creative Sense-Making: Quantifying Interaction Dynamics in Co-Creation

This paper describes a new technique for quantifying interaction dynamics during open-ended co-creation, such as collaborative drawing or playing pretend. We present a cognitive framework called creative sense-making. This framework synthesizes existing cognitive science theories and empirical investigations into open-ended improvisation to develop a method of quantifying cognitive states and types of interactions through time. We apply this framework to empirical studies of human collaboration (in the domain of pretend play) and AI-based systems (in the domain of collaborative drawing) to establish its validity through cross-domain application and inter-rater reliability within each domain. The creative sense-making framework described includes a qualitative coding technique, interaction coding software, and the cognitive theory behind their application.

Quantifying Collaboration with a Co-Creative Drawing Agent

This article describes a new technique for quantifying creative collaboration and applies it to the user study evaluation of a co-creative drawing agent. We present a cognitive framework called creative sense-making that provides a new method to visualize and quantify the interaction dynamics of creative collaboration, for example, the rhythm of interaction, style of turn taking, and the manner in which participants are mutually making sense of a situation. The creative sense-making framework includes a qualitative coding technique, interaction coding software, an analysis method, and the cognitive theory behind these applications. This framework and analysis method are applied to empirical studies of the Drawing Apprentice collaborative sketching system to compare human collaboration with a co-creative AI agent vs. a Wizard of Oz setup. The analysis demonstrates how the proposed technique can be used to analyze interaction data using continuous functions (e.g., integrations and moving averages) to measure and evaluate how collaborations unfold through time.

SolidSketch: Toward Enactive Interactions for Semantic Model Creation

SolidSketch is a solid and parametric modeling program that enables users to rapidly construct 3D parametric and semantic models through sketch and multi-touch input. The interaction design principles of SolidSketch are based on the cognitive science theory of enaction. We argue enactive interactions would support design creativity by enabling rapid iteration and continuous feedback throughout a flexible design exploration. SolidSketch infers the intention of the user by continuously analyzing the surrounding context and users behavior. This paper briefly introduces the enaction theory, the interaction designs as well as the implementations of SolidSketch.