Amit Gruber

Multibody factorization with uncertainty and missing data using the EM algorithm

Multibody factorization algorithms give an elegant and simple solution to the problem of structure from motion even for scenes containing multiple independent motions. Despite this elegance, it is still quite difficult to apply these algorithms to arbitrary scenes. First, their performance deteriorates rapidly with increasing noise. Second, they cannot be applied unless all the points can be tracked in all the frames (as will rarely happen in real scenes). Third, they cannot incorporate prior knowledge on the structure or the motion of the objects. In this paper we present a multibody factorization algorithm that can handle arbitrary noise covariance for each feature as well as missing data. We show how to formulate the problem as one of factor analysis and derive an expectation-maximization based maximum-likelihood algorithm. One of the advantages of our formulation is that we can easily incorporate prior knowledge, including the assumption of temporal coherence. We show that this assumption greatly enhances the robustness of our algorithm and present results on challenging sequences.

Sparse regression as a sparse eigenvalue problem

We extend the l0-norm ldquosubspectralrdquo algorithms developed for sparse-LDA (Moghaddam, 2006) and sparse-PCA (Moghaddam, 2006) to more general quadratic costs such as MSE in linear (or kernel) regression. The resulting ldquosparse least squaresrdquo (SLS) problem is also NP-hard, by way of its equivalence to a rank-1 sparse eigenvalue problem. Specifically, for minimizing general quadratic cost functions we use a highly-efficient method for direct eigenvalue computation based on partitioned matrix inverse techniques that leads to times103 speed-ups over standard eigenvalue decomposition. This increased efficiency mitigates the O(n4) complexity that limited the previous algorithmspsila utility for high-dimensional problems. Moreover, the new computation prioritizes the role of the less-myopic backward elimination stage which becomes even more efficient than forward selection. Similarly, branch-and-bound search for exact sparse least squares (ESLS) also benefits from partitioned matrix techniques. Our greedy sparse least squares (GSLS) algorithm generalizes Natarajanpsilas algorithm (Natarajan, 1995) also known as order-recursive matching pursuit (ORMP). Specifically, the forward pass of GSLS is exactly equivalent to ORMP but is more efficient, and by including the backward pass, which only doubles the computation, we can achieve a lower MSE than ORMP. In experimental comparisons with LARS (Efron, 2004), forward-GSLS is shown to be not only more efficient and accurate but more flexible in terms of choice of regularization.

Incorporating non-motion cues into 3d motion segmentation

We address the problem of segmenting an image sequence into rigidly moving 3D objects. An elegant solution to this problem is the multibody factorization approach in which the measurement matrix is factored into lower rank matrices. Despite progress in factorization algorithms, the performance is still far from satisfactory and in scenes with missing data and noise, most existing algorithms fail. In this paper we propose a method for incorporating 2D non-motion cues (such as spatial coherence) into multibody factorization. We formulate the problem in terms of constrained factor analysis and use the EM algorithm to find the segmentation. We show that adding these cues improves performance in real and synthetic sequences.

Incorporating non-motion cues into 3D motion segmentation

We address the problem of segmenting an image sequence into rigidly moving 3D objects. An elegant solution to this problem in the case of orthographic projection is the multibody factorization approach in which the measurement matrix is factored into lower rank matrices. Despite progress in factorization algorithms, their performance is still far from satisfactory and in scenes with missing data and noise, most existing algorithms fail. In this paper we propose a method for incorporating 2D non-motion cues (such as spatial coherence) into multibody factorization. We show the similarity of the problem to constrained factor analysis and use the EM algorithm to find the segmentation. We show that adding these cues improves performance in real and synthetic sequences.

Incorporating constraints and prior knowledge into factorization algorithms – an application to 3d recovery

Matrix factorization is a fundamental building block in many computer vision and machine learning algorithms. In this work we focus on the problem of structure from motion in which one wishes to recover the camera motion and the 3D coordinates of certain points given their 2D locations. This problem may be reduced to a low rank factorization problem. When all the 2D locations are known, singular value decomposition yields a least squares factorization of the measurements matrix. In realistic scenarios this assumption does not hold: some of the data is missing, the measurements have correlated noise, and the scene may contain multiple objects. Under these conditions, most existing factorization algorithms fail while human perception is relatively unchanged. In this work we present an EM algorithm for matrix factorization that takes advantage of prior information and imposes strict constraints on the resulting matrix factors. We present results on challenging sequences.