Rui F. C. Guerreiro

Estimation of Rank Deficient Matrices from Partial Observations: Two-Step Iterative Algorithms

Several computer vision applications require estimating a rank deficient matrix from noisy observations of its entries. When the observation matrix has no missing data, the LS solution of such problem is known to be given by the SVD. However, in practice, when several entries of the matrix are not observed, the problem has no closed form solution. In this paper, we study two iterative algorithms for minimizing the non-linear LS cost function obtained when estimating rank deficient matrices from partial observations. In the first algorithm, the iterations are the well known Expectation and Maximization (EM) steps that have succeeded in several estimation problems with missing data. The second algorithm, which we call Row-Column (RC), estimates, in alternate steps, the row and column spaces of the solution matrix. Our conclusions are that RC performs better than EM in what respects to the robustness to the initialization and to the convergence speed. We also demonstrate the algorithms when inferring 3D structure from video sequences.

Connectivity-Enforcing Hough Transform for the Robust Extraction of Line Segments

Global voting schemes based on the Hough transform (HT) have been widely used to robustly detect lines in images. However, since the votes do not take line connectivity into account, these methods do not deal well with cluttered images. On the other hand, the so-called local methods enforce connectivity but lack robustness to deal with challenging situations that occur in many realistic scenarios, e.g., when line segments cross or when long segments are corrupted. We address the critical limitations of the HT as a line segment extractor by incorporating connectivity in the voting process. This is done by only accounting for the contributions of edge points lying in increasingly larger neighborhoods and whose position and directional information agree with potential line segments. As a result, our method, which we call segment extraction by connectivity-enforcing HT (STRAIGHT), extracts the longest connected segments in each location of the image, thus also integrating into the HT voting process the usually separate step of individual segment extraction. The usage of the Hough space mapping and a corresponding hierarchical implementation make our approach computationally feasible. We present experiments that illustrate, with synthetic and real images, how STRAIGHT succeeds in extracting complete segments in situations where current methods fail.

3D structure from video streams with partially overlapping images

The majority of methods available to recover 3D structure from video assume that a set of feature points are tracked across a large number of frames. This is not always possible in real videos because the images overlap only partially, due to occlusion and limited field of view. The paper describes a new method to recover 3D structures from videos with partially overlapping views. The well known factorization method recovers 3D rigid structures by factoring an observation matrix that collects the trajectories of feature points (see Tomasi, C. and Kanade, T., Int. J. of Computer Vision, vol.9, no.2, 1992). We extend this method to the more challenging scenario of observing incomplete trajectories. In this way, we accommodate not only features that disappear, but also features that, although not visible in the first image, become available later. Under this scenario, the observation matrix has missing entries. We develop three new algorithms to factor out matrices with missing data. Experiments with synthetic data and real video images demonstrate the viability of our approach to recover 3D structure.

Global motion estimation: feature-based, featureless, or both ?!

The approaches to global motion estimation have been naturally classified into one of two main classes: feature-based methods and direct (or featureless) methods. Feature-based methods compute a set of point correspondences between the images and, from these, estimate the parameters describing the global motion. Although the simplicity of the second step has made this approach rather appealing, the correspondence step is a quagmire and usually requires human supervision. In opposition, featureless methods attempt to estimate the global motion parameters directly from the image intensities, using complex nonlinear optimization algorithms. In this paper, we propose an iterative scheme that combines the feature-based simplicity with the featureless robustness. Our experiments illustrate the behavior of the proposed scheme and demonstrate its effectiveness by automatically building image mosaics.

Incremental local Hough Transform for line segment extraction

Although global voting schemes, such as the Hough Transform (HT), have been widely used to robustly detect lines in images, they fail when the line segments at hand are short, particularly if the underlying edge maps are cluttered. Line segment detection in these scenarios has been addressed using local methods, which lack robustness to missing data (interrupted lines) and typically fail when line segments cross. We propose a new method that tackles these problems: first, rough estimates of plural candidate directions at each edge point are obtained through a directional local HT; then, the parameters determining the line segments are globally estimated by maximizing a quality measure that depends on all the edge points. Our experiments illustrate that the proposed method outperforms current methods in challenging situations.

Factorization with missing data for 3D structure recovery

Matrix factorization methods are now widely used to recover 3D structure from 2D projections [C. Tomasi and T. Kanade. International Journal of Computer Vision, 9(2), 1992] . In this practice, the observation matrix to be factored out has missing data, due to the limited field of view and the occlusion that occur in real video sequences. In opposition to the optimality of the SVD to factor out matrices without missing entries, the optimal solution for the missing data case is not known. In R.F.C. Guerreiro and P.M.Q. Aguiar [IEEE ICIP, New York, USA, September 2002] we introduced suboptimal algorithms that proved to be more efficient than previous approaches to the factorization of matrices with missing data. In this paper we make an experimental analysis of the algorithms of R.F.C. Guerreiro and P.M.Q. Aguiar [IEEE ICIP, New York, USA, September 2002] and demonstrate their performance in virtual reality and video compression applications. We conclude that these algorithms are adequate to the amount of missing entries that may occur when processing real videos; robust to the typical level of noise in practical applications; and computationally as simple as the factorization of matrices without missing entries.

Extraction of line segments in cluttered images via multiscale edges

Current methods for line segment extraction often fail in challenging scenarios that abound in real-life images, e.g., those containing corrupted lines, of various widths, with multiple crossings, and immersed in clutter. We propose a method that tackles these issues by combining multiscale edges while taking line segment connectivity into account. In particular, we use two scales originating what we call contextual and local edges, obtained with filters of, respectively, large and small footprints. Contextual edges are robust to noise and our method uses them validate local edges, i.e., to only select the local edges that correspond to the same intensity transition (dark-to-bright or vice-versa). Line segment connectivity is enforced by joining the valid local edges whose distance does not exceed a threshold. To enable dealing with situations where the edges divide regions of non-uniform intensity distributions, e.g., textures, the contextual edges are decided by using a two-sample statistical test. We present experiments that illustrate how our method is efficient in extracting complete segments in several situations where current methods fail.

Optimized filters for efficient multi-texture discrimination

When performing texture analysis via standard filter banks, good discrimination depends on the usage of a large number of filters. For example, when using the popular Gabor Filter Banks, the typical number of filters ranges from about ten to fifty. For applications requiring high frame rate processing, this is too complex. Also, discrimination may be poor if the method is not adapted to the characteristics of the target textures. Optimized filters attempt to solve these issues by automatically creating filters that are tuned to the target textures. Although these filters have shown to perform well when the number of textures to discriminate is small, their computational complexity increases dramatically in situations that arise in practice, e.g., those exhibiting ten or more classes of textures. In this paper, we propose optimized filters for efficient multi-texture discrimination. In particular, we propose two alternative filter designs: one-dimensional filters, applied horizontally and vertically, for orientation-dependent discrimination; and ring-shaped filters for rotationally invariant discrimination. Texture classification is based on the first four moments of the filter outputs, which are simple to compute yet approximate more sophisticated methods. The filter parameters are tuned through supervised learning, which is performed by using a Genetic Algorithm, that deals well with the non-convex nature of the objective function. We test our method with the Brodatz and VisTex albums, concluding that it outperforms state-of-the-art methods in terms of computational simplicity and accuracy.

Learning simple texture discrimination filters

Current texture analysis methods enable good discrimination but are computationally too expensive for applications which require high frame rates. This occurs because they use redundant calculations, failing in capturing the essence of the texture discrimination problem. In this paper we use a learning approach to obtain simple filters for this task. Although others have proposed learning-based methods, we are the first to simultaneously achieve discrimination rates comparable with state-of-the art methods at high frame rates. We particularize the general methodology to different filter structures, e.g., rotationally discriminant filters and rotationally invariant ones. We use Genetic Algorithms for learning and test our method against state-of-the-art ones, using the Brodatz album.