José Ignacio Ronda

Stochastic rate-control of video coders for wireless channels

We introduce a new approach to deal with the transmission of real-time video over wireless channels, based on a priori stochastic models for both source and channel. This new problem formulation captures in a natural way the stochastic nature of the channel as well as the uncertainty regarding the properties of the video sequence. Our formulation leads to an optimal control problem that can be solved off-line, employing standard stochastic dynamic programming techniques. The outcome of this optimization is an off-line control policy that is optimal in the sense of minimizing the average coding distortion. The on-line computational cost of the new approach is thus very low: all that is required during run-time is to identify the state of the system (source and channel). Unlike other optimization-based rate control techniques, which require a search for the optimal operating point, the operating points here for each allowable state of the system have been precalculated. We consider wireless packet-based transmission with Automatic Repeat reQuest (ARQ) error control. While a standard model has been adopted to characterize the channel behavior, a new model based on the concept of coding complexity has been devised in order to characterize the video source. Simulation results based on this new approach are provided and compared to other proposed rate-control strategies. They show how the use of model-based optimal policies has negligible on-line computational cost while providing a transmission quality comparable to that achieved with more costly deterministic dynamic programming techniques, and significantly better than for simpler algorithms that do not explicitly take into account the channel state.

The Absolute Line Quadric and Camera Autocalibration

Abstract.We introduce a geometrical object providing the same information as the absolute conic: the absolute line quadric (ALQ). After the introduction of the necessary exterior algebra and Grassmannian geometry tools, we analyze the Grassmannian of lines of P from both the projective and Euclidean points of view. The exterior algebra setting allows then to introduce the ALQ as a quadric arising very naturally from the dual absolute quadric. We fully characterize the ALQ and provide clean relationships to solve the inverse problem, i.e., recovering the Euclidean structure of space from the ALQ. Finally we show how the ALQ turns out to be particularly suitable to address the Euclidean autocalibration of a set of cameras with square pixels and otherwise varying intrinsic parameters, providing new linear and non-linear algorithms for this problem. We also provide experimental results showing the good performance of the techniques.

Line Geometry and Camera Autocalibration

We provide a completely new rigorous matrix formulation of the absolute quadratic complex (AQC), given by the set of lines intersecting the absolute conic. The new results include closed-form expressions for the camera intrinsic parameters in terms of the AQC, an algorithm to obtain the dual absolute quadric from the AQC using straightforward matrix operations, and an equally direct computation of a Euclidean-upgrading homography from the AQC. We also completely characterize the 6×6 matrices acting on lines which are induced by a spatial homography.Several algorithmic possibilities arising from the AQC are systematically explored and analyzed in terms of efficiency and computational cost. Experiments include 3D reconstruction from real images.

Photorealistic 3D reconstruction from handheld cameras

One of the major challenges in the fields of computer vision and computer graphics is the construction and representation of life-like virtual 3D scenarios within a computer. The VISIRE project attempts to reconstruct photo-realistic 3D models of large scenarios using as input multiple freehand video sequences, while rendering the technology accessible to the non-expert.VISIRE is application oriented and hence must deal with multiple issues of practical relevance that were commonly overlooked in past experiences. The paper presents both an innovative approach for the integration of previously unrelated experiences, as well as a number of novel contributions, such as: an innovative algorithm to enforce closedness of the trajectories, a new approach to 3D mesh generation from sparse data, novel techniques dealing with partial occlusions and a method for using photo-consistency and visibility constrains to refine the 3D mesh.

Advanced rate control for MPEG-4 coders

The interest developed in the last years on the coded representations of video signals allowing independent manipulation of semantic picture elements has become one of the guidelines for the new ISO standard MPEG-4. The real- time generation of a bit-stream according to this new paradigm, which can be described as the multiplexing of a set of independently coded, arbitrarily shaped video objects, posies new requirements regarding the transmission through either fixed or variable-rate channels.In particular, adequate control algorithms should support the encoding of objects with different quality requirements and be robust with respect to rapid changes in size and shape of the objects. After formalizing this new rate-control problem in terms consistent with the previous requirements, this paper focuses on the design of the corresponding rate- control algorithms in the real-time case, introducing and evaluating an approach relying on the modelization of the source and the application of optimization criteria based on rate-distortion concepts for the tuning of the different object qualities. The experimental work presented, which corresponds to the recent MPEG-4 video verification model coder specification, apart from making apparent the superiority of the global control over the individual control of the coding of each object, allows for a comparison of the advantages and inconveniences of the different optimization objectives.

Camera Autocalibration and Horopter Curves

We describe a new algorithm for the obtainment of the affine and Euclidean calibration of a camera under general motion. The algorithm exploits the relationships of the horopter curves associated to each pair of cameras with the plane at infinity and the absolute conic. Using these properties we define cost functions whose minimization by means of general purpose techniques provides the required calibration. The experiments show the good convergence properties, computational efficiency and robust performance of the new techniques.

VISIRE: photorealistic 3D reconstruction from video sequences

Traditionally, building 3D reconstructions of large scenarios such as a museum or historical site has been costly, time consuming and required the contribution of expert personnel. Usually the results showed an artificial look and had little interactivity. However, newly developed technologies in the areas of video analysis, camera calibration and texture fusion allow us to think in a much more satisfying scenario where the user with the only aid of a domestic video camera is able to acquire all the information it is required to construct the 3D model of the desired environment in an easy and comfortable manner. In this paper, the results obtained in the EC funded project VISIRE are presented. VISIRE attempts to construct photorealistic 3D models of large scenarios using as input multiple freehand video sequences. Once acquired, the computer vision software processes the video information off-line in order to obtain the 3D mesh together with the textures required to obtain a 3D model highly resembling the original.

Evaluation of DWT and DCT for irregular mesh-based motion compensation in predictive video coding

While traditional video coding standards employ block-based processing (regarding motion estimation, motion compensation and DCT) which produce undesired artifacts, the most recent developments (H.263, MPEG) include first improvements in motion compensation (such as overlapped block motion compensation), which constitute a first step to avoid these limitations. Mesh-based motion compensation represents a further progress in this area as it treats the motion information in a continuous way over the whole frame. We investigated in these methods and have found that especially triangle meshes over irregularly spread node points lead to very good results. We presented first results of the compensation step, here we will show results of the complete coding scheme for rectangular frames. As the insertion of the transform/coding part raised the problem of which type of transformation should be employed to code the error image efficiently and without disturbing the gains of the compensation step, we here compare DCT and DWT (discrete wavelet transform) with different filter types, to find out which one is most useful to apply with mesh-based motion compensation.

Stochastic rate-control of interframe video coders for VBR channels

We propose a new algorithm for the real-time control of an inter-frame video coder operating with a variable rate channel such as wireless channels or the Internet. Using techniques of stochastic dynamic programming we obtain off-line optimal policies from stochastic models of the channel and coder which minimize the average expected distortion. The on-line complexity of our approach is only that required to identify the state of the system (source and channel). The state of the channel is obtained based on the ARQ error-control mechanism, and the source state is computed as complexity measurements on each incoming frame. Simulation results based on this new approach are provided and compared to other proposed rate-control strategies. They show how our model-based optimal policies outperform the other considered approaches keeping a negligible on-line computational cost. This result is very interesting when considering an alternative to traditional costly solutions based on deterministic dynamic programming.

Linear camera autocalibration with varying parameters

We provide a new technique for the Euclidean upgrading of a projective calibration for a set of ten or more cameras with known skew angle and aspect ratio and arbitrary varying focal length and principal point. The proposed algorithm, which is purely linear and thus of very low computational cost and not suffering from initialization problems, is based on the geometric object given by the set of lines incident with the absolute conic. We include experiments, which show the good performance of the technique.