Tommi Ilmonen

Martial arts in artificial reality

This paper presents Kick Ass Kung-Fu, a martial arts game installation where the player fights virtual enemies with kicks and punches as well as acrobatic moves such as cartwheels. Using real-time image processing and computer vision, the video image of the user is embedded inside 3D graphics. Compared to previous work, our system uses a profile view and two displays, which allows an improved view of many martial arts techniques. We also explore exaggerated motion and dynamic slow-motion effects to transform the aesthetic of kung-fu movies into an interactive, embodied experience. The system is described and analyzed based on results from testing the game in a theater, in a television show, and in a user study with 46 martial arts practitioners.

Extending large-scale event participation with user-created mobile media on a public display

Most large public displays have been used for providing information to passers-by with the primary purpose of acting as one-way information channels to individual users. We have developed a large public display to which users can send their own media content using mobile devices. The display supports multi-touch interaction, thus enabling collaborative use of the display. This display called CityWall was set up in a city center with the goal of showing information of events happening in the city. We observed two user groups who used mobile phones with upload capability during two large-scale events happening in the city. Our findings are that this kind of combined use of personal mobile devices and a large public display as a publishing forum, used collaboratively with other users, creates a unique setting that extends the groups feeling of participation in the events. We substantiate this claim with examples from user data.

Comedia: mobile group media for active spectatorship

Previous attempts to support spectators at large-scale events have concentrated separately on real-time event information, awareness cues, or media-sharing applications. CoMedia combines a group media space with event information and integrates reusable awareness elements throughout. In two field trials, one at a rally and the other at a music festival, we found that CoMedia facilitated onsite reporting to offsite members, coordination of group action, keeping up to date with others, spectating remotely, and joking. In these activities, media, awareness cues, and event information were often used in concert, albeit assuming differing roles. We show that the integrated approach better supports continuous interweaving of use with the changing interests and occurrences in large-scale events.

Implementation issues of 3D audio in a virtual room

This paper presents the audio system built for the virtual room at Helsinki University of Technology. First we discuss the general problems for multichannel sound reproduction caused by the construction of, and the equipment in virtual rooms. We also describe the acoustics of the room in question, and the effect of the back-projected screens and reverberation to the sound. Compensation of the spectral deficiencies and the problems with the large listening area and high frequency attenuation are introduced. The hardware configuration used for sound reproduction is shortly described. We also report the software applications and libraries built for sound signal processing and 3D sound reproduction.

Realtime audiovisual rendering and contemporary audiovisual art

Visual rendering is the process of creating synthetic images of digital models. The modelling of sound synthesis and propagation in a virtual space is called sound rendering. In this article we review different audiovisual rendering techniques suitable for realtime rendering of three-dimensional virtual worlds. Virtual environments are useful in various application areas, for example in architectural visualisation. With audiovisual rendering, lighting and acoustics of a modelled concert hall can be experienced early in the design stage of the building. In this article we demonstrate an interactive audiovisual rendering system where an animated virtual orchestra plays in a modelled concert hall. Virtual musicians are conducted by a real conductor who wears a wired data dress suit and a baton. The conductor and the audience hear the music rendered according to the acoustics of the virtual concert hall, creating a lifelike experience.

Extreme Filters-cache-efficient implementation of long IIR and FIR filters

Modern general purpose processors have powerful vector processing units that can be used in infinite impulse response and finite impulse response filtering in real time. In practice, the calculations are efficient only if the filter parameters fit into the processors cache. With long filters, the computation is slowed down since the filter data do not fit into the cache at once. By computing the filter in multiple segments, more efficient cache utilization can be achieved. The presented optimization method works when multiple samples are processed in row. The improved cache efficiency increases performance by up to almost one magnitude over direct filter implementation on modern hardware

Period detection and representation by recurrent oscillatory self-organizing map

Whether or not periodicity is a property of the environment, for a cognitive system a period is always a dynamical mental construct. This study suggests a dynamical implementation of this hypothesis using recurrent oscillatory self-organizing map of the feature space of such streams. The mapping allows the system to assign identity and class-membership to each stream point, represented by a locus on a two-dimensional map. The model is shown to detect periodicities of various regular and behavior-originated, single- and multi-channel wave patterns, and to reproduce such signals, relying solely on the oscillating activation of the units.

Evaluating an automatic rotation feature in collaborative tabletop workspaces

Tabletops are commonly used for collaboration but would benefit from features that help orient objects to individual users disposed around the display. We propose an approach of automatic orientation based on fingers and hand detection as a proxy to determine the position of the user. To contribute to the discussion of the relevance of automatic rotation, we present a comparison study of pairs of participants engaged in both loosely and tightly coupled tasks. We collected performance measures, questionnaires and analyze interactions from video recordings. The results show that automatic rotation is more suitable when the collaboration is loosely coupled. Conversely, in tightly coupled tasks performance are worse and user ratings low when automatic rotations are enabled. We conclude that features such as automatic orientation on tabletop are important and promising but that they need to be critically assessed with respect to their effects on collaboration in both tightly and loosely coupled tasks.

Upponurkka: An Inexpensive Immersive Display for Public VR Installations

Upponurkka is an inexpensive immersive display system for public virtual reality installations. It is designed for public places to be robust, wireless, and light-weight. The system consists of two passive stereo screens and optical tracking. In this paper the hardware and software of Upponurkka are briefly presented. In addition, experiences of using the system in a public art exhibition are reported.

Virtual concerts in virtual spaces—in real time

The DIVA system is an experimental interactive real‐time virtual environment with synchronized sound and animation components. The system provides real‐time automatic character animation and visualization, dynamic behavior control of virtual actors, interaction through motion analysis, sound generation with physical models of musical instruments, and three‐dimensional sound auralization. The combined effect of 3‐D visual and acoustic elements creates stronger immersion than would be possible with either alone. As a demonstration, a virtual band with four artificial musicians has been implemented. The user interacts with the virtual musicians by showing the tempo with a baton, like real conductors do. The animated band follows the gestures of the conductor and another user controls the viewpoint of the audience. Due to the real‐time acoustic modeling and sound rendering, both users hear the auralized music in a real concert hall.