Ivana Tosic

Learning Sparse Representations of Depth

This paper introduces a new method for learning and inferring sparse representations of depth (disparity) maps. The proposed algorithm relaxes the usual assumption of the stationary noise model in sparse coding. This enables learning from data corrupted with spatially varying noise or uncertainty, such as that obtained by laser range scanners or structured light depth cameras. Sparse representations are learned from the Middlebury database disparity maps and then exploited in a two-layer graphical model for inferring depth from stereo, by including a sparsity prior on the learned features. Since they capture higher order dependencies in the depth structure, these priors can complement smoothness priors commonly used in depth inference based on Markov random field (MRF) models. Inference on the proposed graph is achieved using an alternating iterative optimization technique, where the first layer is solved using an existing MRF-based stereo matching algorithm, then held fixed as the second layer is solved using the proposed nonstationary sparse coding algorithm. This leads to a general method for improving solutions of state-of-the-art MRF-based depth estimation algorithms. Our experimental results first show that depth inference using learned representations leads to state-of-the-art denoising of depth maps obtained from laser range scanners and a time of flight camera. Furthermore, we show that adding sparse priors improves the results of depth estimation methods based on graph cut optimization of MRFs with first and second order priors.

3D face recognition using sparse spherical representations

This paper addresses the problem of 3D face recognition using spherical sparse representations. We first propose a fully automated registration process that permits to align the 3D face point clouds. These point clouds are then represented as signals on the 2D sphere, in order to take benefit of the geometry information. Simultaneous sparse approximations implement a dimensionality reduction process by subspace projection. Each face is typically represented by a few spherical basis functions that are able to capture the salient facial characteristics. The dimensionality reduction step preserves the discriminant facial information and eventually permits an effective matching in the reduced space, where it can further be combined with LDA for improved recognition performance. We evaluate the 3D face recognition algorithm on the FRGC v.1.0 data set, where it outperforms classical state-of-the-art solutions based on PCA or LDA on depth face images.

Ultrasound tomography with learned dictionaries

We propose a new method for imaging sound speed in breast tissue from measurements obtained by ultrasound tomography (UST) scanners. Given the measurements, our algorithm finds a sparse image representation in an overcomplete dictionary that is adapted to the properties of UST images. This dictionary is learned from high resolution MRI breast scans using an unsupervised maximum likelihood dictionary learning method. The proposed dictionary-based regularization method significantly improves the quality of reconstructed breast UST images. It outperforms the wavelet-based reconstruction and the least squares minimization with lowpass constraints, on both numerical and in vivo data. Our results demonstrate that the use of the learned dictionary improves the image accuracy for up to 4 dB with the exact measurement matrix and for 3.5 dB with the estimated measurement matrix over the wavelet-based reconstruction under the same conditions.

Dictionary Learning for Stereo Image Representation

One of the major challenges in multi-view imaging is the definition of a representation that reveals the intrinsic geometry of the visual information. Sparse image representations with overcomplete geometric dictionaries offer a way to efficiently approximate these images, such that the multi-view geometric structure becomes explicit in the representation. However, the choice of a good dictionary in this case is far from obvious. We propose a new method for learning overcomplete dictionaries that are adapted to the joint representation of stereo images. We first formulate a sparse stereo image model where the multi-view correlation is described by local geometric transforms of dictionary elements (atoms) in two stereo views. A maximum-likelihood (ML) method for learning stereo dictionaries is then proposed, where a multi-view geometry constraint is included in the probabilistic model. The ML objective function is optimized using the expectation-maximization algorithm. We apply the learning algorithm to the case of omnidirectional images, where we learn scales of atoms in a parametric dictionary. The resulting dictionaries provide better performance in the joint representation of stereo omnidirectional images as well as improved multi-view feature matching. We finally discuss and demonstrate the benefits of dictionary learning for distributed scene representation and camera pose estimation.

3D face recognition with sparse spherical representations

This paper addresses the problem of 3D face recognition using simultaneous sparse approximations on the sphere. The 3D face point clouds are first aligned with a fully automated registration process. They are then represented as signals on the 2-sphere in order to preserve depth and geometry information. Next, we implement a dimensionality reduction process with simultaneous sparse approximations and subspace projection. It permits to represent each 3D face by only a few spherical functions that are able to capture the salient facial characteristics, and hence to preserve the discriminant facial information. We eventually perform recognition by effective matching in the reduced space, where linear discriminant analysis can be further activated for improved recognition performance. The 3D face recognition algorithm is evaluated on the FRGC v.1.0 data set, where it is shown to outperform classical state-of-the-art solutions that work with depth images.

Geometry-Based Distributed Scene Representation With Omnidirectional Vision Sensors

This paper addresses the problem of efficient representation of scenes captured by distributed omnidirectional vision sensors. We propose a novel geometric model to describe the correlation between different views of a 3-D scene. We first approximate the camera images by sparse expansions over a dictionary of geometric atoms. Since the most important visual features are likely to be equivalently dominant in images from multiple cameras, we model the correlation between corresponding features in different views by local geometric transforms. For the particular case of omnidirectional images, we define the multiview transforms between corresponding features based on shape and epipolar geometry constraints. We apply this geometric framework in the design of a distributed coding scheme with side information, which builds an efficient representation of the scene without communication between cameras. The Wyner-Ziv encoder partitions the dictionary into cosets of dissimilar atoms with respect to shape and position in the image. The joint decoder then determines pairwise correspondences between atoms in the reference image and atoms in the cosets of the Wyner-Ziv image in order to identify the most likely atoms to decode under epipolar geometry constraints. Experiments demonstrate that the proposed method leads to reliable estimation of the geometric transforms between views. In particular, the distributed coding scheme offers similar rate-distortion performance as joint encoding at low bit rate and outperforms methods based on independent decoding of the different images.

Learning Joint Intensity-Depth Sparse Representations

This paper presents a method for learning overcomplete dictionaries of atoms composed of two modalities that describe a 3D scene: 1) image intensity and 2) scene depth. We propose a novel joint basis pursuit (JBP) algorithm that finds related sparse features in two modalities using conic programming and we integrate it into a two-step dictionary learning algorithm. The JBP differs from related convex algorithms because it finds joint sparsity models with different atoms and different coefficient values for intensity and depth. This is crucial for recovering generative models where the same sparse underlying causes (3D features) give rise to different signals (intensity and depth). We give a bound for recovery error of sparse coefficients obtained by JBP, and show numerically that JBP is superior to the group lasso algorithm. When applied to the Middlebury depth-intensity database, our learning algorithm converges to a set of related features, such as pairs of depth and intensity edges or image textures and depth slants. Finally, we show that JBP outperforms state of the art methods on depth inpainting for time-of-flight and Microsoft Kinect 3D data.

Progressive Coding of 3-D Objects Based on Overcomplete Decompositions

This paper presents a progressive coding scheme for 3-D objects, based on overcomplete signal expansions on the 2-D sphere. Due to increased freedom in the basis construction, redundant expansions have shown interesting approximation properties in the decomposition of signals with multidimensional singularities organized along embedded submanifolds. We propose to map simple 3-D models on 2-D spheres and then to decompose the signal over a redundant dictionary of oriented and anisotropic atoms living on the sphere. The signal expansion is computed iteratively with a matching pursuit algorithm, which greedily selects the most prominent components of the 3-D model. The decomposition therefore inherently represents a progressive stream of atoms, which is advantageously used in the design of scalable representations. An encoder is proposed that compresses the stream of atoms by adaptive coefficient quantization and entropy coding of atom indexes. Experimental results show that the novel coding strategy outperforms state-of-the-art progressive coders in terms of distortion, mostly at low bit rates. Furthermore, since the dictionary is built on structured atoms, the proposed representation simultaneously offers an increased flexibility for easy stream manipulations. We finally illustrate that advantage in the design of a view-dependent transmission scheme

Low bit-rate compression of omnidirectional images

Omnidirectional images represent a special type of images that are captured by vision sensors with a 360-degree field of view. This work targets the compression of such images by taking into account their particular geometry. We first map omnidirectional images to spherical ones and then perform sparse image decomposition over a dictionary of geometric atoms on the 2D sphere. A coder based on Matching Pursuit and adaptive quantization is finally proposed for efficient compression of omnidirectional images. The experiments demonstrate that the proposed system outperforms JPEG2000 coding of unfolded images. Since most omnidirectional sensors can be parametrized with a spherical camera model, the proposed method is generic with respect to different sensor constructions.

Distributed Coding of Multiresolution Omnidirectional Images

This paper addresses the problem of compact representation of a 3D scene, captured by distributed omnidirectional cameras. As the images from the sensors are likely to be correlated in most practical scenarios, we build a distributed algorithm based on coding with side information. A reference image is processed with a wavelet transform and progressively encoded. The Wyner-Ziv images undergo a multiresolution representation, and the generated bitplanes are channel encoded with LDPC codes. The central decoder eventually reconstructs the Wyner-Ziv images given by the syndrome bits from the channel codes using the reference omnidirectional image. It also iteratively implements motion estimation on the 2-sphere in order to improve the side information. Experimental results demonstrate that distributed coding improves the rate-distortion performance for coding a set of omnidirectional images when compared to independent coding solutions. The proposed method can further be extended to the decoding of multiple Wyner-Ziv images using one single reference omnidirectional image. Hence, it achieves a reduced overall coding rate compared to disparity-based schemes. In addition, it does not require explicit knowledge of the camera parameters nor precise calibration, which is certainly interesting in camera networks.