Masahito Shiba

A CG projection method of supporting to stage live performances

Recently many live performances use projected visual images and videos. Our goal is to provide a supporting system for the creative use of CG animation for live performances. This paper describes a method of supporting live stage performances by projecting computer-created visual images. We developed a system to project images in a theatre, and then staged performances with both living actors and virtual CG animations. The system can project images in various types of theatres, and can adapt to the trial and error of rehearsals, not constricted to pre-set programs in actual performances. This paper describes our experience with one of our performances featuring collaborative scenes of actual actors with virtual actors created by CG with human motion-data. Moreover we created transitions representing paradise and various levels of hell. After the performances, we received comments from director and spectators that the projection of CG animations was a fascinating and powerful method for combining live performances and CG.

A system for motion synthesis of human body parts using a touch panel

This paper describes a support system for choreography creation utilizing data on human body motion archived by motion capture systems. Users of the system can synthesize and compose choreographies using a touch panel device. The users set the motions of each body part to synthesize, and then the synthesized motion is displayed as CG animation in real-time. The system calculates the positions of the floor and body parts and detects inappropriate synthesis motion. The system prevents the creation of inappropriate motions, and thus the users can only create natural motions.

Choreography composition and live performance on a Noh stage

We have been developing a simulation system for dance using a 3D motion archive. Our goal is to provide a supporting system for the creative use of motion data for live performances. We used motion data and this system on a Noh stage, which is normally used by the classic Japanese masked musical drama. The performance was a new Noh based on Shakespeares Macbeth. One challenge in this performance was to create collaborative scenes of real and virtual actors. We displayed virtual actors on a projection screen. We deployed two real actors and one virtual actor as three witches in the Macbeth story. Another challenge was real-time composition of dance on stage. We controlled one of the virtual actors to dance according to the real actors on the stage. In this paper, we describe the method of choreography composition and our experience using the system in a Noh theater.

A virtual performance system and its application on a Noh stage

We have developed a virtual performance system that projects visual images and controls them in a theater easily. This system manages videos and 3D models, and users can control the projected image in real-time. We used the system on a Noh stage. We displayed virtual actors and had them act with the real actors. As a result of evaluations by the audience, we verified that using virtual actors acting with real actors can provide effective stage presentations.

An attempt of real-time CG control with multi-touch devices

We have been developing real-time CG control systems with Lemurs, which are multi-touch devices. We have developed two prototype systems that control CG objects and animation and a practical system that supports creating of TV contents. Our system, which provides an easy and intuitive way to control two or more parameters simultaneously, allows users to control such complicated data as real-time CG contents. We verified that the system can be used in actual broadcasting.

Enhancement of Stage Performances by Projecting CG Animation and Control of Images with Sensors

We have been producing live stage performances using computer graphics (CG) and interactive techniques. We developed a system to support and enhance live performances. This system projects CG onstage alongside real actors and allows users to control the projected images in real time. Collaborative scenes between real actors and virtual actors created by CG are possible. The system has multi-functions and is flexible, hence, it can be used for purposes other than performances. The system helps directors and actors in the creation of performances, as well as during rehearsals. Moreover, the system is portable and easy to set up in various types of theaters. The system can use sensor devices to determine physical expressions of actors or spectators. These perceived states can be used to affect projected images, which can provide a richer presentation. This paper describes the structure of the system and its effects in live performances that we attempted.

Enhancement of stage performances with projected CG and Noh motions

Our goal is to provide a support system for creating rich and flexible presentations in live performances using motion data. We created a live stage performance and a projection system which is for enhancing the performance with videos and CG animations. We choreographed and captured Noh motions, and created animation of CG characters to project. The CG characters were deployed in a responsive manner during a dance performance and integrated with live actors. In this paper, we describe the CG creation using Noh motion data, and our experience using the projection system and the CG in the live stage performance.

Digital Archive and Exhibiting Methods of a Buddhist Ceremonial Procession

Our research group has been archiving a Buddhist ceremonial procession called Nerikuyo. Nerikuyo has special features in its walking movements and actions. It is difficult to display these features using traditional panels in a museum. Our purpose is to create videos and interactive content that vividly describe this ceremony. We have archived the ceremony with super-high-detail videos and then created video contents for a special exhibition on Nerikuyo. We have also proposed a virtual fitting system that recognizes poses or gestures of users and then displays the corresponding images and sounds over the captured images of the users. All of the gestures are related to the poses or motions of Nerikuyo, and they are assigned to masks and tools. The created videos were shown at a special exhibition of the Ryukoku Museum, and the proposed system was demonstrated for three days as one of the related events of the special exhibition.

A Method of Managing Distributed Contents in Heterogeneous Servers

Various types of content servers are used to run digital archives and digital museums in many sites. These content servers have different interfaces and ways to keep their contents. Inappropriately managed content servers prevent efficient use of contents in different servers. This paper proposes a solution to this inefficiency by describing a peer-to-peer based content management system that integrates distributed contents in two or more content servers. Our proposed system has a mechanism for controlling content servers and creating content control data for managing contents. The proposed system manages the content servers and the contents with these data over a peer-to-peer network. With this management method, the proposed system provides scalable and efficient use of the contents by integrating the servers and the contents.