Andrew M. Webb

combinFormation: a mixed-initiative system for representing collections as compositions of image and text surrogates

We develop combinFormation, a mixed-initiative system for representing collections as compositions of image and text surrogates. The system provides a set of direct manipulation facilities for forming, editing, organizing, and distributing collections as compositions. Additionally, to assist users in sifting through the vast expanse of potentially relevant information resources, the system also includes a generative agent that can proactively engage in processes of collecting information resources and forming image and text surrogates. A generative temporal visual composition agent develops the collection and its visual representation over time, enabling users to see more possibilities. To keep the user in control, we develop interactive techniques that enable the user to direct the agent. For evaluation, we conducted a field study in an undergraduate general education course offered in the architecture department. Alternating groups of students used combinFormation as an aid in preparing one of two major assignments involving information discovery to support processes of invention. The students that used combinFormation were found to perform better

combinFormation: Mixed-initiative composition of image and text surrogates promotes information discovery

combinFormation is a mixed-initiative creativity support tool for searching, browsing, organizing, and integrating information. Images and text are connected to represent surrogates (enhanced bookmarks), optimizing the use of human cognitive facilities. Composition, an alternative to lists and spatial hypertext, is used to represent a collection of surrogates as a connected whole, using principles from art and design. This facilitates the creative process of information discovery, in which humans develop new ideas while finding and collecting information. To provoke the user to think about the large space of potentially relevant information resources, a generative agent proactively engages in collecting information resources, forming image and text surrogates, and composing them visually. The agent develops the collection and its visual representation over time, enabling the user to see ideas and relationships. To keep the human in control, we develop interactive mechanisms for authoring the composition and directing the agent. In a field study in an interdisciplinary course on The Design Process, over a hundred students alternated using combinFormation and Google+Word to collect prior work on information discovery invention assignments. The students that used combinFormations mixed-initiative composition of image and text surrogates performed better.

Using Metrics of Curation to Evaluate Information-Based Ideation

Evaluating creativity support environments is challenging. Some approaches address people’s experiences of creativity. The present method measures creativity, across conditions, in the products that people make. This research introduces information-based ideation (IBI), a paradigm for investigating open-ended tasks and activities in which users develop new ideas. IBI tasks span imagining, planning, and reflecting on a weekend, vacation, outfit, makeover, paper, internship, thesis, design, campaign, crisis response, career, or invention. What products do people create through engagement in IBI? Curation of digital media incorporates conceptualization, finding and choosing information objects, annotation, and synthesis. Through engagement in IBI tasks, people create curation products. This article formulates a quantitative methodology for evaluating IBI support tools, building on prior creative cognition research in engineering design to derive a battery of ideation metrics of curation. Elemental ideation metrics evaluate creativity within curated found objects. Holistic ideation metrics evaluate how a curation puts elements together. IBI support environments are characterized by their underlying medium of curation. Curation media include lists, such as listicles, and grids, such as the boards of Pinterest. An in-depth case study investigates information composition, an art-based medium representing a curation as a freeform visual semantic connected whole. We raise two creative cognition challenges for IBI. One challenge is overcoming fixation—for instance, when a person gets stuck in a counterproductive mental set. The other challenge is to bridge information visualization’s synthesis gap, by providing support for connecting findings. To address the challenges, we develop mixed-initiative information composition (MI2C), integrating human curation of information composition with automated agents of information retrieval and visualization. We hypothesize that MI2C generates provocative stimuli that help users overcome fixation to become more creative on IBI tasks. We hypothesize that MI2C’s integration of curation and visualization bridges the synthesis gap to help users become more creative. To investigate these hypotheses, we apply ideation metrics of curation to interpret results from experiments with 44 and 49 participants.

Promoting reflection and interpretation in education: curating rich bookmarks as information composition

Reflection, interpretation, and curation play key roles in learning, creativity, and problem solving. Reflection means looking back and forward among building blocks constituting a space of ideas, contextualizing with processes including tasks, activities, and ones internal thinking and meditating, and deriving new understandings, known as interpretations. Curation, in the digital age, means searching, gathering, collecting, organizing, designing, reflecting on, and interpreting information. We introduce rich bookmarks, representations of key ideas from documents as navigable links that integrate visual clippings and rich semantic metadata. We support curating rich bookmarks as information composition. In this holistic visual form, curators express relationships among curated elements through implicit visual features, such as spatial position, color, and translucence. We investigated the situated context of a university course, engaging educators in iterative co-design. Rich bookmarks emerged in the process, motivating changes in pedagogy and software. Changes provoked students to collect more novel and varied ideas. They reported that curating rich bookmarks as information composition helped them reflect, transforming prior ideas into new ones. The visual component of rich bookmarks was found to support multiple interpretations; the semantic to support associational exploration of related ideas.

Meta-metadata: a metadata semantics language for collection representation applications

Collecting, organizing, and thinking about diverse information resources is the keystone of meaningful digital information experiences, from research to education to leisure. Metadata semantics are crucial for organizing collections, yet their structural diversity exacerbates problems of obtaining and manipulating them, strewing end users and application developers amidst the shadows of a proverbial tower of Babel. We introduce meta-metadata, a language and software architecture addressing a metadata semantics lifecycle: (1) data structures for representation of metadata in programs; (2) metadata extraction from information resources; (3) semantic actions that connect metadata to collection representation applications; and (4) rules for presentation to users. The language enables power users to author metadata semantics wrappers that generalize template-based information sources. The architecture supports development of independent collection representation applications that reuse wrappers. The initial meta-metadata repository of information source wrappers includes Google, Flickr, Yahoo, IMDb, Wikipedia, and the ACM Portal. Case studies validate the approach.

Choreographic buttons: promoting social interaction through human movement and clear affordances

We used human movement as the basis for designing a collaborative aesthetic design environment. Our intention was to promote social interaction and creative expression. We employed off-the-shelf computer vision technology. Movement became the basis for the choreography of gestures, the development of gesture recognition, and the development of imagery and visualization. We discovered that the design of clear affordances is no less important in movement-based than in mouse-based systems. Through an integrated and iterative design process, we developed a new type of affordance, the choreographic button, which integrates choreography, gesture recognition, and visual feedback. Jumping, a quick movement, and crouching, a sustained gesture, were choreographed to form a vocabulary that is personally expressive, and which also facilitates automatic recognition.How can we evaluate socially motivated interactive systems? To create a context for evaluation, we held an integrated exhibition, party, and user study event. This mixing of events produced an engaging environment in which participants could choose to interact with each other, as well as with the design environment. We prepared a mouse-based version of the design environment, and compared how people experienced it with the movementbased system. Our study demonstrates that movement-based affordances promote social interaction.

Distributed Liveness: Understanding How New Technologies Transform Performance Experiences

We identify emerging phenomena of distributed liveness, involving new relationships among performers, audiences, and technology. Liveness is a recent, technology-based construct, which refers to experiencing an event in real-time with the possibility for shared social realities. Distributed liveness entails multiple forms of physical, spatial, and social co-presence between performers and audiences across physical and virtual spaces. We interviewed expert performers about how they experience liveness in physically co-present and distributed settings. Findings show that distributed performances and technology need to support flexible social co-presence and new methods for sensing subtle audience responses and conveying engagement abstractly.

Patterns of Free-form Curation: Visual Thinking with Web Content

Web curation involves choosing, organizing, and commenting on content. Popular web curation apps-- e.g., Facebook, Twitter, and Pinterest-- provide linear feeds that show people the latest content, but provide little support for articulating relationships among content elements. The new medium of free-form web curation enables multimedia elements to be spontaneously gathered from the web, written about, sketched amidst, manipulated, and visually assembled in a continuous space. Through free-form web curation, content is collected, interpreted, and arranged, creating context. We conducted a field study of 1581 students in 6 courses, spanning diverse fields. We derive patterns of free-form curation through a visual grounded theory analysis of the resulting dataset of 4426 curations. From the observed range of invocations of the patterns in the performance of ideation tasks, we conclude that free-form is valuable as a new medium of web curation in how it supports creative visual thinking.

Beyond Slideware: How a Free-form Presentation Medium Stimulates Free-form Thinking in the Classroom

We investigate how presentation in a free-form medium stimulates free-form thinking and discussion in the classroom. Most classroom presentations utilize slideware (e.g. PowerPoint). Yet, slides add intrusive segregations that obstruct the flow of information. In contrast, in a free-form medium of presentation, content is not separated into rigid slide compartments. Instead, it is visually arranged and transformed in a continuous space. We develop a case study that investigates student experiences authoring, presenting, viewing, and discussing free-form presentations in a graduate seminar class. We analyze interviews, present a sampling of student presentations, and develop findings: free-form presentation stimulates free-form thinking, spontaneous discussion, and emergent ideation.

Generating views of the buzz: browsing popular media and authoring using mixed-initiative composition

combinFormations mixed-initiative composition space enables system agents and humans to engage in processes of finding relevant information, and forming and authoring collections. Previously, the system was developed and utilized to support information discovery. The efficacy of the system for supporting creativity has been established in some contexts. We present combinFormation as a tool for browsing popular media and authoring personal collections. Yahoo Buzz is an popular media collection of top search queries, categorized into genres such as actors, music and sports. Existing interfaces limit the human participant to only viewing results of a single search at any given time. This paper presents a new system structure which interleaves multiple searches concurrently in a round-robin manner, enabling participants to concurrently explore and connect diverse result sets, which, in aggregate, may consist of hundreds of documents. The mixed-initiative composition space serves as a media interface for combining search results and authoring personal collections. Evaluation using the Buzz demonstrated participants ability to browse more diverse information using combinFormation than with a typical browser. They experienced browsing and authoring as easier and more entertaining. Results have implications for a broad range of use contexts in which combined views of the results of multiple searches need to be authored, including research scenarios, as well as popular media.