Francisco Morán

3D graphics rendering time modeling and control for mobile terminals

3D graphics has found its way to mobile devices such as Personal Digital Assistants (PDA) and cellular phones. Given their limited battery capabilities, these devices typically have less computational resources available than their counterparts connected to a power supply. Additionally, the workload of 3D graphics applications changes very drastically over time. These different and changing conditions make the creation of 3D content a real challenge for the content creators.To allow the rendering of arbitrary content on a mobile device without the need of ad-hoc content creation. We present a framework to adapt the resolution of 3D objects to the available processing resources. An MPEG-4 scalable geometry decoder is used to change the resolution and an analytical model of the workload of a mobile renderer is presented for controlling the scalable decoder. Because of the scarce computational resources, a good balance between accuracy and complexity is needed. The presented approach has an error and a complexity overhead of less than 10% for most practical cases.

Progressive lower trees of wavelet coefficients: efficient spatial and SNR scalable coding of 3d models

We perform an in-depth analysis of current state-of-the-art waveletbased 3D model coding techniques and then present a new one that outperforms them in terms of compression efficiency and, more importantly, provides full spatial and SNR scalability: PLTW (Progressive Lower Tree Wavelet) coding. As all SNR scalable bit-streams, ours can be used in heterogeneous networks with a wide range of terminals, both in terms of processing power and bandwidth. But because of being spatially scalable, the PLTW bit-stream does not impose on the less powerful terminals the need of building detail trees as deep as required by the maximum LOD, because the wavelet coefficients are sent on a per-LOD basis, thus achieving a “local” SNR scalability within a “global” spatial scalability. In particular, we show that our technique provides a substantial advantage over the only similar one in a current ISO standard (MPEG-4), and thus suggest that PLTW be considered for its future versions.

Moving object detection for real-time augmented reality applications in a GPGPU

The last generation of consumer electronic devices is endowed with Augmented Reality (AR) tools. These tools require moving object detection strategies, which should be fast and efficient, to carry out higher level object analysis tasks. We propose a lightweight spatio-temporal-based non-parametric background-foreground modeling strategy in a General Purpose Graphics Processing Unit (GPGPU), which provides real-time high-quality results in a great variety of scenarios and is suitable for AR applications.

GPU-based implementation of an optimized nonparametric background modeling for real-time moving object detection

Answering to the growing demand of computer vision tools for the last generations of consumer electronic devices equipped with smart cameras, several nonparametric moving detection algorithms have been developed. These algorithms, by modeling both background and foreground from spatio-temporal reference data, provide satisfactory results in many complex scenarios. However, to be computationally efficient, they apply some simplifications that decrease the quality of the detections. This paper presents a novel real-time implementation of an optimized spatio-temporal nonparametric moving object detection strategy. To improve the quality of previous algorithms, the bandwidths of the kernels required to model the background are dynamically estimated, and the background model is also selectively updated. The proposed implementation features smart cooperation between a computer/devices Central and Graphics Processing Units (CPU/GPU) and extensive usage of the texture mapping and filtering units of the latter, including a novel method for fast evaluation of Gaussian functions. Thanks to these features, high quality detection rates are achieved while respecting the realtime restrictions imposed by computer vision tools running on current consumer electronic devices.

A model for adapting 3D graphics based on scalable coding, real-time simplification and remote rendering

Most current multiplayer 3D games can only be played on dedicated platforms, requiring specifically designed content and communication over a predefined network. To overcome these limitations, the OLGA (On-Line GAming) consortium has devised a framework to develop distributive, multiplayer 3D games. Scalability at the level of content, platforms and networks is exploited to achieve the best trade-offs between complexity and quality.Most current multiplayer 3D games can only be played on dedicated platforms, requiring specifically designed content and communication over a predefined network. To overcome these limitations, the OLGA (On-Line GAming) consortium has devised a framework to develop distributive, multiplayer 3D games. Scalability at the level of content, platforms and networks is exploited to achieve the best trade-offs between complexity and quality.

Subdivision surfaces in MPEG-4

SS (subdivision surfaces) are a powerful modelling paradigm for truly hierarchical (instead of merely progressive) 3D surface (as opposed to mesh) coding. Two SS-based tools of the MPEG-4 animation framework extension allow us to derive a piecewise smooth surface from an initial control mesh: if the subdivision process is run in its predefined form, the initial mesh is simply smoothed; if 3D details are added to the positions of the new vertices appearing after each subdivision step, a particular target surface may be approximated with an increasing accuracy. In both cases, multiresolution editing/animation is possible.

3D game content distributed adaptation in heterogeneous environments

Most current multiplayer 3D games can only be played on a single dedicated platform (a particular computer, console, or cell phone), requiring specifically designed content and communication over a predefined network. Below we show how, by using signal processing techniques such as multiresolution representation and scalable coding for all the components of a 3D graphics object (geometry, texture, and animation), we enable online dynamic content adaptation, and thus delivery of the same content over heterogeneous networks to terminals with very different profiles, and its rendering on them. We present quantitative results demonstrating how the best displayed quality versus computational complexity versus bandwidth tradeoffs have been achieved, given the distributed resources available over the end-to-end content delivery chain. Additionally, we use state-of-the-art, standardised content representation and compression formats (MPEG-4 AFX, JPEG 2000, XML), enabling deployment over existing infrastructure, while keeping hooks to well-established practices in the game industry.

Seamless, Static Multi-Texturing of 3D Meshes

In the context of 3D reconstruction, we present a static multi‐texturing system yielding a seamless texture atlas calculated by combining the colour information from several photos from the same subject covering most of its surface. These pictures can be provided by shooting just one camera several times when reconstructing a static object, or a set of synchronized cameras, when dealing with a human or any other moving object. We suppress the colour seams due to image misalignments and irregular lighting conditions that multi‐texturing approaches typically suffer from, while minimizing the blurring effect introduced by colour blending techniques. Our system is robust enough to compensate for the almost inevitable inaccuracies of 3D meshes obtained with visual hull–based techniques: errors in silhouette segmentation, inherently bad handling of concavities, etc.

Static 3D triangle mesh compression overview

3D triangle meshes are extremely used to model discrete surfaces, and almost always represented with two tables: one for geometry and another for connectivity. While the raw size of a triangle mesh is of around 200 bits per vertex, by coding cleverly (and separately) those two distinct kinds of information it is possible to achieve compression ratios of 15:1 or more. Different techniques must be used depending on whether single-rate vs. progressive bitstreams are sought; and, in the latter case, on whether or not hierarchically nested meshes are desirable during reconstruction.

Authoring 744: first results

This paper presents the first results of the Authoring744 research initiative, which uses MPEG-7 to synthesize MPEG-4 content. The objective is to use MPEG-7 content descriptions to synthesize content, instead of creating descriptions by analyzing existing content. The output uses MPEG-4 XMT as the representation format, which is further used to create an MPEG-4 binary format, which can in turn be played.