Nikolce Stefanoski

Coding Algorithms for 3DTV—A Survey

Research efforts on 3DTV technology have been strengthened worldwide recently, covering the whole media processing chain from capture to display. Different 3DTV systems rely on different 3D scene representations that integrate various types of data. Efficient coding of these data is crucial for the success of 3DTV. Compression of pixel-type data including stereo video, multiview video, and associated depth or disparity maps extends available principles of classical video coding. Powerful algorithms and open international standards for multiview video coding and coding of video plus depth data are available and under development, which will provide the basis for introduction of various 3DTV systems and services in the near future. Compression of 3D mesh models has also reached a high level of maturity. For static geometry, a variety of powerful algorithms are available to efficiently compress vertices and connectivity. Compression of dynamic 3D geometry is currently a more active field of research. Temporal prediction is an important mechanism to remove redundancy from animated 3D mesh sequences. Error resilience is important for transmission of data over error prone channels, and multiple description coding (MDC) is a suitable way to protect data. MDC of still images and 2D video has already been widely studied, whereas multiview video and 3D meshes have been addressed only recently. Intellectual property protection of 3D data by watermarking is a pioneering research area as well. The 3D watermarking methods in the literature are classified into three groups, considering the dimensions of the main components of scene representations and the resulting components after applying the algorithm. In general, 3DTV coding technology is maturating. Systems and services may enter the market in the near future. However, the research area is relatively young compared to coding of other types of media. Therefore, there is still a lot of room for improvement and new development of algorithms.

Connectivity-Guided Predictive Compression of Dynamic 3D Meshes

We introduce an efficient algorithm for real-time compression of temporally consistent dynamic 3D meshes. The algorithm uses mesh connectivity to determine the order of compression of vertex locations within a frame. Compression is performed in a frame to frame fashion using only the last decoded frame and the partly decoded current frame for prediction. Following the predictive coding paradigm, local temporal and local spatial dependencies between vertex locations are exploited. In this framework we present a novel angle preserving predictor and evaluate its performance against other state of the art predictors. It is shown that the proposed algorithm improves up to 25% upon the current state of the art for compression of temporally consistent dynamic 3D meshes.

Scalable Linear Predictive Coding of Time-Consistent 3D Mesh Sequences

We present a linear predictive compression approach for time-consistent 3D mesh sequences supporting and exploiting scalability. The algorithm decomposes each frame of a mesh sequence in layers employing patch-based mesh simplification techniques. This layered decomposition is consistent in time. Following the predictive coding paradigm, local temporal and spatial dependencies between layers and frames are exploited for compression. Prediction is performed vertex-wise from coarse to fine layers exploiting the motion of already encoded 1-ring neighbor vertices for prediction of the current vertex location. It is shown that a predictive exploitation of the proposed layered configuration of vertices can improve the compression performance upon other state-of-the-art approaches by up to 16% in domains relevant for applications.

Spatially and temporally scalable compression of animated 3D meshes with MPEG-4 / FAMC

We introduce an efficient method for scalable compression of animated 3D meshes. The approach consists of a combination of inter-frame motion compensation and layer-wise predictive coding of remaining residuals. It is shown that motion compensated predictive coding can lead to a significant improvement in compression performance upon the current state of the art with gains of over 40%. In addition, the created bit stream is temporally and spatially scalable allowing an adaptation to network transfer rates and end-user devices. This compression method is currently standardized within MPEG as part of MPEG-4 AFX Amd. 2, where it is referred to as FAMC-frame-based animated mesh compression.

SPC: Fast and Efficient Scalable Predictive Coding of Animated Meshes

Animated meshes are often represented by a sequence of static meshes with constant connectivity. Due to their frame‐based representation they usually occupy a vast amount of bandwidth or disk space. We present a fast and efficient scalable predictive coding (SPC) scheme for frame‐based representations of animated meshes. SPC decomposes animated meshes in spatial and temporal layers which are efficiently encoded in one pass through the animation. Coding is performed in a streamable and scalable fashion. Dependencies between neighbouring spatial and temporal layers are predictively exploited using the already encoded spatio‐temporal neighbourhood. Prediction is performed in the space of rotation‐invariant coordinates compensating local rigid motion. SPC supports spatial and temporal scalability, and it enables efficient compression as well as fast encoding and decoding. Parts of SPC were adopted in the MPEG‐4 FAMC standard. However, SPC significantly outperforms the streaming mode of FAMC with coding gains of over 33%, while in comparison to the scalable FAMC, SPC achieves coding gains of up to 15%. SPC has the additional advantage over FAMC of achieving real‐time encoding and decoding rates while having only low memory requirements. Compared to some other non‐scalable state‐of‐the‐art approaches, SPC shows superior compression performance with gains of over 16% in bit‐rate.

The New MPEG-4/FAMC Standard for Animated 3D Mesh Compression

This paper presents a new compression technique for 3D dynamic meshes, referred to as FAMC - Frame-based Animated Mesh Compression, recently promoted within the MPEG-4 standard as Amen-dement 2 of part 16 (AFX -Animation Framework extension). The FAMC approach combines a model-based motion-compensation strategy with transform/predictive coding of residual errors. First, a skinning motion-compensation model is automatically derived from a frame-based representation. Subsequently, either 1) DCT/lifting wavelets or 2) layer-based predictive coding is employed to exploit remaining spatio-temporal correlations in the residual signal. Both motion model parameters and residual signal components are finally encoded by using context-based adaptive binary arithmetic coding (CABAC). The proposed FAMC encoder offers high compression performance with gains of 60% in terms of bit-rate savings relative to previous MPEG-4 technology and of 20% to 40% relative to state-of-the-art techniques. FAMC is well suited for compressing both geometric and photometric (normal vectors, colors...) attributes. In addition, FAMC also supports a rich set of functionalities including streaming, scalability (spatial, temporal and quality) and progressive transmission.

Frame-based compression of animated meshes in MPEG-4

This paper presents a new compression technique for 3D dynamic meshes, referred to as FAMC - frame-based animated mesh compression, promoted within the MPEG-4 standard as amendment 2 of part 16 AFX (animation framework extension). The FAMC approach combines a model-based motion compensation strategy, with transform/predictive coding of residual errors. First, a skinning motion compensation model is automatically computed from a frame-based representation and then encoded. Subsequently, either 1) DCT/lifting wavelets or 2) layer-based predictive coding is employed to exploit remaining spatio-temporal correlations in the residual signal. The proposed encoder offers high compression performances (gains in bit rate of 60% with respect to the previous MPEG-4 technique and of 20% to 40% with respect to state-of-the-art approaches) and is well suited for compressing both geometric and photometric (normal vectors, colors...) attributes. In addition, the FAMC method supports a rich set of functionalities including streaming, scalability (spatial, temporal and quality) and progressive transmission.

Layered Predictive Coding of Time-Consistent Dynamic 3D Meshes using a Non-Linear Predictor

We present a layered predictive compression approach for time-consistent dynamic 3D meshes. The algorithm decomposes each frame of a dynamic 3D mesh in layers employing patch-based mesh simplification techniques. This layered decomposition is consistent in time. Following the predictive coding paradigm, local temporal and spatial dependencies between layers and frames are exploited for compression. Prediction is performed vertex-wise from coarse to fine layers exploiting local linear and non-linear dependencies between vertex locations for compression. It is shown that a non-linear predictive exploitation of the proposed layered configuration of vertices can improve the compression performance upon other state-of-the-art approaches by more than 15% in domains relevant for applications.

Depth estimation and depth enhancement by diffusion of depth features

Current trends in video technology indicate a significant increase in spatial and temporal resolution of video data. Recently, a linear-runtime feature diffusion algorithm was presented which aims for fast and accurate processing of such high resolution data. In this paper, we introduce this algorithm from the perspective of image-based depth estimation, expanding upon the algorithm by requiring interview consistency in the depth diffusion process. We also discuss different application scenarios and provide an in-depth analysis of the method in this context.

MIXED POISSON APPROXIMATION OF NODE DEPTH DISTRIBUTIONS IN RANDOM BINARY SEARCH TREES

We investigate the distribution of the depth of a node containing a specific key or, equivalently, the number of steps needed to retrieve an item stored in a randomly grown binary search tree. Using a representation in terms of mixed and compounded standard distributions, we derive approximations by Poisson and mixed Poisson distributions; these lead to asymptotic normality results. We are particularly interested in the influence of the key value on the distribution of the node depth. Methodologically our message is that the explicit representation may provide additional insight if compared to the standard approach that is based on the recursive structure of the trees. Further, in order to exhibit the influence of the key on the distributional asymptotics, a suitable choice of distance of probability distributions is important. Our results are also applicable in connection with the number of recursions needed in Hoares [Comm. ACM 4 (1961) 321-322] selection algorithm Find.