Tinsley A. Galyean

Multi-level direction of autonomous creatures for real-time virtual environments

There have been several recent efforts to build behavior-based autonomous creatures. While competent autonomous action is highly desirable, there is an important need to integrate autonomy with “directability”. In this paper we discuss the problem of building autonomous animated creatures for interactive virtual environments which are also capable of being directed at multiple levels. We present an approach to control which allows an external entity to “direct” an autonomous creature at the motivational level, the task level, and the direct motor level. We also detail a layered architecture and a general behavioral model for perception and action-selection which incorporates explicit support for multi-level direction. These ideas have been implemented and used to develop several autonomous animated creatures.

Sculpting: an interactive volumetric modeling technique

We present a new interactive modeling technique based on the notion of sculpting a solid material. A sculpting tool is controlled by a 3D input device and the material is represented by voxel data; the tool acts by modifying the values in the voxel array, much as a paint programs paintbrush modifies bitmap values. The voxel data is converted to a polygonal surface using a marching-cubes algorithm; since the modifications to the voxel data are local, we accelerate this computation by an incremental algorithm and accelerate the display by using a special data structure for determining which polygons must be redrawn in a particular screen region. We provide a variety of tools: one that cuts away material, one that adds material, a sandpaper tool, a heat gun, etc. The technique provides an intuitive direct interaction, as if the user were working with clay or wax. The models created are free-form and may have complex topology; however, they are not precise, so the technique is appropriate for modeling a boulder or a tooth but not for modeling a crankshaft.

Rendering interactive holographic images

We present a method for computing holographic patterns for the generation of three-dimensional (3-D) holographic images at interactive speeds. We used this method to render holograms on a conventional computer graphics workstation. The framebuffer system supplied signals directly to a real-time holographic (“holovideo”) display. We developed an efficient algorithm for computing an image-plane stereogram, a type of hologram that allowed for several computational simplifications. The rendering algorithm generated the holographic pattern by compositing a sequence of view images that were rendered using a recentering shear-camera geometry. Computational efficiencies of our rendering method allowed the workstation to calculate a 6-megabyte holographic pattern in under 2 seconds, over 100 times faster than traditional computing methods. Data-transfer time was negligible. Holovideo displays are ideal for numerous 3-D visualization applications, and promise to provide 3-D images with extreme realism. Although the focus of this work was on fast computation for holovideo, the computed holograms can be displayed using other holographic media. We present our method for generating holographic patterns, preceded by a background section containing an introduction to optical and computational holography and holographic displays.

Guided navigation of virtual environments

This paper presents a new methodology for navigating virtual environments called “The River Analogy.” This analogy provides a new way of thinking about the users relationship to the virtual environment; guiding the users continuous and direct input within both space and time allowing a more narrative presentation. The paper then presents the details of how this analogy was applied to a VR experience that is now part of the permanent collection at the Chicago Museum of Science and Industry.

CINEMA: a system for procedural camera movements

This paper presents a general system for camera movement upon which a wide variety of higher-level methods and applications can be built. In addition to the basic commands for camera placement, a key attribute of the CINEMA system is the ability to inquire information directly about the 3D world through which the camera is moving. With this information high-level procedures can be written that closely correspond to more natural camera specifications. Examples of some high-level procedures are presented. In addition, methods for overcoming deficiencies of this procedural approach are proposed.

Multi-level Control for Animated Autonomous Agents: Do the Right Thing...Oh, Not That..

Autonomous animated characters can play an important role in Interactive Story Systems by freeing the director from the details of time-step to time-step control. However, to be truly useful, the director must be able to control the character at a number of different levels of abstraction, and levels of influence. We have presented a system which shows how these ideas may be integrated into an architecture for autonomous animated characters.

Recursive estimation for CAD model recovery

We describe a system for semiautomatically extracting 3-D object models from raw, uncalibrated video. The system utilizes a recursive estimator to accurately recover camera motion, point-wise structure, and camera focal length. Recovered 3-D points are used to compute a piecewise-smooth surface model for the object. Recovered motion and camera geometry are then used along with the original video to texture map the surfaces. We describe extensions to our previously-reported geometry estimation formulation that incorporate focal length estimation and other improvements, so that accurate estimates of structure and camera motion can be recovered from uncalibrated video cameras. We also discuss the buildup of texture maps from sequences of images, which is important in producing realistic looking models. Examples demonstrate generation of a realistic 3-D texture mapped model from a video sequence, the post-production manipulation of video, and the combination of computer graphics models with video.<<ETX>>

Virtual FishTank

Participants can: • Create their own fish. • Design behaviors for their fish. • Observe their fish interacting with other fish. • Manipulate behavioral rules for a group of fish. • Discover how these behaviors can emulate schooling. • Analyze emerging patterns. Through real-time 3D graphics, visitors are introduced to ideas from the sciences of complexity – ideas that explain not only ecosystems, but also economic markets, immune systems, and traffic jams. In particular, visitors learn how complex patterns arise from simple rules. The first version of Virtual FishTank opens at The Computer Museum in Boston in June 1998. A second version will travel nationally to other science museums and aquariums.

Mobile Devices for Early Literacy Intervention and Research with Global Reach

Extensive work focuses on the uses of technology at scale for post-literate populations (e.g., MOOC, learning games, Learning Management Systems). Little attention is afforded to non-literate populations, particularly in the developing world. This paper presents an approach using mobile devices with the ultimate goal to reach 770 million people. We developed a novel platform with a cloud backend to deliver educational content to over a thousand marginalized children in different countries: specifically, in remote villages without schools, urban slums with overcrowded schools, and at-risk, rural schools. Here we describe the theoretical basis of our system and results from case studies in three educational contexts. This model will help researchers and designers understand how mobile devices can help children acquire basic skills and aid each others learning when the benefit of teachers is limited or non-existent.

“I Hold Your Foot:” Lessons from the Reading Brain for Addressing the Challenge of Global Literacy

In this chapter we describe the theoretical and technological principles that underlie an innovative application of cross-disciplinary work in cognitive neuroscience, linguistics, child development, education, and technology: The global literacy initiative, Curious Learning. We will delineate the basic principles about the reading brain from cognitive neuroscience that guided the content of our work, and the principles from technology that guided design, implementation, and data collection. This initiative represents our ongoing efforts to develop and implement a comprehensive, tablet-based digital learning experience to help children learn to read on their own, particularly those children who either possess no school or whose schools are so inadequate that the children never achieve functional literacy.