Frank Nielsen

HYPERMASK TALKING HEAD PROJECTED ONTO REAL OBJECT

HyperMask is a system which projects an animated face onto a physical mask, worn by an actor. As the mask moves within a prescribed area, its position and orientation are detected by a camera, and the projected image changes with respect to the viewpoint of the audience. The lips of the projected face are automatically synthesized in real time with the voice of the actor, who also controls the facial expressions. As a theatrical tool, HyperMask enables a new style of storytelling. As a prototype system, we propose to put a self-contained HyperMask system in a trolley (disguised as a linen cart), so that it projects onto the mask worn by the actor pushing the trolley. Keyword: Talking Face, Mask Tracking, Facial Expression Synthesis, Lip Synchronization, Interactive Entertainment, HYPERMASK TALKING HEAD PROJECTED ONTO REAL OBJECT SHIGEO MORISHIMA AND TATSUO YOTSUKURA Faculty of Engineering, Seikei University 3-3-1 Kichijoji-Kitamachi, Musashino, Tokyo 180-8633 JAPAN E-mail: {shigeo, yotsu}@ee.seikei.ac.jp KIM BINSTED AND FRANK NIELSEN Sony Computer Science Laboratories 3-14-13 Higashi-Gotanda Shinagawa-ku Tokyo 141 E-mail: {kimbm, nielsen}@csl.sony.co.jp CLAUDIO PINHANEZ IBM T.J. Watson Research 30 Saw Mill River Rd. (Route 9A) Hawthorne, NY 10532 E-mail: pinhanez@us.ibm.com HYPERMASK is a system which projects an animated face onto a physical mask, worn by an actor. As the mask moves within a prescribed area, its position and orientation are detected by a camera, and the projected image changes with respect to the viewpoint of the audience. The lips of the projected face are automatically synthesized in real time with the voice of the actor, who also controls the facial expressions. As a theatrical tool, HYPERMASK enables a new style of storytelling. As a prototype system, we propose to put a self-contained HYPERMASK system in a trolley (disguised as a linen cart), so that it projects onto the mask worn by the actor pushing the trolley.

Surround video: a multihead camera approach

AbstractWe describe algorithms for creating, storing and viewing high-resolution immersive surround videos. Given a set of unit cameras designed to be almost aligned at a common nodal point, we first present a versatile process for stitching seamlessly synchronized streams of videos into a single surround video corresponding to the video of the multihead camera. We devise a general registration process onto raymaps based on minimizing a tailored objective function. We review and introduce new raymaps with good sampling properties. We then give implementation details on the surround video viewer and present experimental results on both real-world acquired and computer-graphics rendered full surround videos. We conclude by mentioning potential applications and discuss ongoing related activities. Video supplements: http://www.csl.sony.co.jp/person/nielsen

HyperMask --- projecting a talking head onto a real object

1 Faculty of Engineering, Seikei University, 3-3-1 Kichijoji-Kitamachi, Musashino-shi, Toyko 180-8633, Japan E-mail: {yotsu,shigeo}@ee.seikei.ac.jp 2 Sony Computer Science Laboratories Inc., 3-14-13 Higashi-Gotanda, Shinagawa-ku, Tokyo 141-0022, Japan E-mail: nielsen@csl.sony.co.jp 3 I-chara Inc., 2-34-1 Uehara, Shibuya-ku, Tokyo 151-0064, Japan E-mail: kimb@i-chara.com 4 IBM Research, Watson, Route 134, P.O. Box 218, Yorktown Heights, N.Y. 10598, USA E-mail: pinhanez@us.ibm.com

Maintenance of a piercing set for intervals with applications

We show how to maintain efficiently a minimum piercing set for a set S of intervals on the line, under insertions and deletions to/from S. A linear-size dynamic data structure is presented, which enables us to compute a new minimum piercing set following an insertion or deletion in time O(c( S) log |S|), where c (S) is the size of the new minimum piercing set. We also show how to maintain a piercing set for S of size at most (1+ɛ)c (S), for 0 < ɛ ≤ 1 , in

CLAIRVOYANCE: A Fast And Robust Precision Mosaicing System for Gigapixel Images

\bar O

Emerging Markets During the Crisis

((log |S|)/ɛ) amortized time per update. We then apply these results to obtain efficient solutions to the following three problems: (i) the shooter location problem, (ii) computing a minimum piercing set for arcs on a circle, and (iii) dynamically maintaining a box cover for a d -dimensional point set.

Image processing apparatus, image processing method, and computer program

In this paper, we present CLAIRVOYANCE: a fully automatic photomosaicing system for building ultra high-resolution composited images from image sequences captured by tailored in-house motorized active pan-tilt digital camera units. Our stand-alone mobile systems built over the past five years are computationally fast, robust to various datasets and deliver unprecedented consumer-level image quality. We describe our simple yet novel lens calibration and radiometric correction procedures based on a fast block matching algorithm. All of our core image stitching components are based on the 2D Fourier phase correlation principle, and are thus easily amenable to hardware LSI implementation. We validate our approach by presenting sharp photomosaics obtained from a few hundreds up to a few thousands data sets of images

Non-flat image processing apparatus, image processing method, recording medium, and computer program

The 2008 economic crisis has offered another look at how emerging-market stocks have behaved relative to developed markets. In the aftermath of the crisis, the authors of this article take a fresh look at emerging markets to explore these questions: Have emerging markets matched growth forecasts? Which segments have performed well? How have emerging markets behaved relative to developed markets? While in the aggregate emerging market stocks were not immune to the crisis, there were some clear differences between emerging and developed markets in the performance of particular sectors and styles.