Fabio Bettio

A Large Scale Interactive Holographic Display

Our work focuses on the development of interactive multi-user holographic displays that allow freely moving naked eye participants to share a three dimensional scene with fully continuous, observer independent, parallax. Our approach is based on a scalable design that exploits a specially arranged array of projectors and a holographic screen. The feasibility of such an approach has already been demonstrated with a working hardware and software 7.4M pixel prototype driven at 10-15Hz by two DVI streams. In this short contribution, we illustrate our progress, presenting a 50M pixel display prototype driven by a dedicated cluster hosting multiple consumer level graphic cards.

A Scalable Hardware and Software System for the Holographic Display of Interactive Graphics Applications

We present a scalable holographic system design targeting multi-user interactive computer graphics applications. The display uses a specially arranged array of micro-displays and a holographic screen. Each point of the holographic screen emits light beams of different color and intensity to the various directions, in a controlled manner. The light beams are generated through a light modulation system arranged in a specic geometry and the holographic screen makes the necessary optical transformation to compose these beams into a perfectly continuous 3D view. With proper software control, the light beams leaving the various pixels can be made to propagate in multiple directions, as if they were emitted from physical objects at xed spatial locations. The display is driven by DVI streams generated by multiple consumer level graphics boards and decoded in real-time by image processing units that feed the optical modules at high refresh rates. An OpenGL compliant library running on a client PC redenes the OpenGL behavior to multicast graphics commands to server PCs, where they are re-interpreted for implementing holographic rendering. The feasibility of the approach has been successfully evaluated with a working hardware and software 7.4M pixel prototype driven at 10-15Hz by three DVI streams.

IsoCam: Interactive Visual Exploration of Massive Cultural Heritage Models on Large Projection Setups

We introduce a novel user interface and system for exploring extremely detailed 3D models in a museum setting. Three-dimensional models and associated information are presented on a large projection surface controlled by a touch-enabled surface placed at a suitable distance in front of it. Our indirect user interface, dubbed IsoCam, combines an object-aware interactive camera controller with an interactive point-of-interest selector and is implemented within a scalable implementation based on multiresolution structures shared between the rendering and user interaction subsystems. The collision-free camera controller automatically supports the smooth transition from orbiting to proximal navigation, by exploiting a distance-field representation of the 3D object. The point-of-interest selector exploits a specialized view similarity computation to propose a few nearby easily reachable interesting 3D views from a large database, move the camera to the user-selected point of interest, and provide extra information through overlaid annotations of the target view. The capabilities of our approach have been demonstrated in a public event attended by thousands of people, which were offered the possibility to explore submillimetric reconstructions of 38 stone statues of the Mont’e Prama Nuragic complex, depicting larger-than-life human figures, and small models of prehistoric Nuraghe (cone-shaped stone towers). A follow-up of this work, using 2.5m-high projection screens, is now included in permanent exhibitions at two Archeological Museums. Results of a thorough user evaluation, involving quantitative and subjective measurements, are discussed.

A point-based system for local and remote exploration of dense 3D scanned models

We present a client-server framework for network distribution and real-time point-based rendering of large 3D models on commodity graphics platforms. Model inspection, based on a one-touch interface, is enriched by a bidirectional hyperlink system which provides access to multiple layers of multimedia contents linking different parts of the 3D model many information sources. In addition to view and light control, users can perform simple 3D operations like angle, distance and area measurements on the 3D model. An authoring tool derived from the basic client allows users to add multimedia content to the model description. Our rendering method is based on a coarse grained multiresolution structure, where each node contains thousands of point samples. At runtime, a view-dependent refinement process incrementally updates the current GPU-cached model representation from local or remote out-of-core data. Vertex and fragment shaders are used for high quality elliptical sample drawing and a variety of shading effects. The system is demonstrated with examples that range from documentation and inspection of small artifacts to exploration of large sites, in both a museum and a large scale distribution setting.

Mont’e Scan: Effective Shape and Color Digitization of Cluttered 3D Artworks

We propose an approach for improving the digitization of shape and color of 3D artworks in a cluttered environment using 3D laser scanning and flash photography. To separate clutter from acquired material, semiautomated methods are employed to generate masks used to segment the range maps and the color photographs. This approach allows the removal of unwanted 3D and color data prior to the integration of acquired data in a 3D model. Sharp shadows generated by flash acquisition are easily handled by this masking process, and color deviations introduced by the flash light are corrected at the color blending step by taking into account the geometry of the object. The approach has been evaluated in a large-scale acquisition campaign of the Mont’e Prama complex. This site contains an extraordinary collection of stone fragments from the Nuragic era, which depict small models of prehistoric nuraghe (cone-shaped stone towers), as well as larger-than-life archers, warriors, and boxers. The acquisition campaign has covered 37 statues mounted on metallic supports. Color and shape were acquired at a resolution of 0.25mm, which resulted in more than 6,200 range maps (about 1.3G valid samples) and 3,817 photographs.

An interactive 3D medical visualization system based on a light field display

We present a prototype medical data visualization system exploiting a light field display and custom direct volume rendering techniques to enhance understanding of massive volumetric data, such as CT, MRI, and PET scans. The system can be integrated with standard medical image archives and extends the capabilities of current radiology workstations by supporting real-time rendering of volumes of potentially unlimited size on light field displays generating dynamic observer-independent light fields. The system allows multiple untracked naked-eye users in a sufficiently large interaction area to coherently perceive rendered volumes as real objects, with stereo and motion parallax cues. In this way, an effective collaborative analysis of volumetric data can be achieved. Evaluation tests demonstrate the usefulness of the generated depth cues and the improved performance in understanding complex spatial structures with respect to standard techniques.

A Practical Vision Based Approach to Unencumbered Direct Spatial Manipulation in Virtual Worlds

We present a practical approach for developing interactive environments that allows humans to interact with large complex 3D models without them having to manually operate input devices. The system provides support for scene manipulation based on hand tracking and gesture recognition and for direct 3D interaction with the 3D models in the display space if a suitably registered 3D display is used. Being based on markerless tracking of a user’s two hands, the system does not require users to wear any input or output devices. 6DOF input is provided by using both hands simultaneously, making the tracker more robust since only tracking of position information is required. The effectiveness of the method is demonstrated with a simple application for model manipulation on a large stereo display, in which rendering constraints are met by employing state-of-the-art multiresolution techniques.

An interactive multi-user holographic environment

We present an interactive multi-user holographic environment that allows freely moving naked eye participants to share a large 3D scene with fully continuous, observer independent, parallax.

Scalable rendering of massive triangle meshes on light field displays

We report on a multiresolution rendering system driving light field displays based on a specially arranged array of projectors and a holographic screen. The system gives multiple freely moving naked-eye viewers the illusion of seeing and manipulating 3D objects with continuous viewer-independent parallax. Multiresolution techniques which take into account the displayed light field geometry are employed to dynamically adapt model resolution to display capabilities and timing constraints. The approach is demonstrated on two different scales: a desktop PC driving a 7.4Mbeams TV-size display, and a cluster-parallel solution driving a large ( 1.6 i? 0.9 m ) 35Mbeams display which supports a room-size working space. In both cases, massive meshes of tens of millions of triangles are manipulated at interactive rates.

Digital Mont'e Prama: 3D Cultural Heritage presentations in museums and anywhere.

We present an interactive visualization system developed for the valorization of an extraordinary collection of protostoric Mediterranean sculptures, which depict models of buildings (cone-shaped stone towers), as well as larger-than-life human figures. The architecture is based on scalable components for efficient distribution and adaptive rendering of extremely detailed surface meshes, as well as a simple and effective interactive camera controller tailored for touch interaction. The user interface has been designed for targeting both small screens and large display systems, and in a way that casual users can easily and naturally explore the models with fast learning curves. Furthermore, a thumbnail-based point-of-interest selector enable users to explore 3D views with information presented as 2D overlays decorating the 3D scene. The system components have been integrated in different interactive applications, ranging from large-screen museal setups and low end mobile devices both with very high visual quality. The capabilities of the museal systems have been demonstrated in a variety of temporal and permanent exhibitions, where they have been extensively used by tens of thousands of visitors.