Mustafa Ayazoglu

Activity recognition using dynamic subspace angles

Cameras are ubiquitous everywhere and hold the promise of significantly changing the way we live and interact with our environment. Human activity recognition is central to understanding dynamic scenes for applications ranging from security surveillance, to assisted living for the elderly, to video gaming without controllers. Most current approaches to solve this problem are based in the use of local temporal-spatial features that limit their ability to recognize long and complex actions. In this paper, we propose a new approach to exploit the temporal information encoded in the data. The main idea is to model activities as the output of unknown dynamic systems evolving from unknown initial conditions. Under this framework, we show that activity videos can be compared by computing the principal angles between subspaces representing activity types which are found by a simple SVD of the experimental data. The proposed approach outperforms state-of-the-art methods classifying activities in the KTH dataset as well as in much more complex scenarios involving interacting actors.

Dynamic subspace-based coordinated multicamera tracking

This paper considers the problem of sustained multicamera tracking in the presence of occlusion and changes in the target motion model. The key insight of the proposed method is the fact that, under mild conditions, the 2D trajectories of the target in the image planes of each of the cameras are constrained to evolve in the same subspace. This observation allows for identifying, at each time instant, a single (piecewise) linear model that explains all the available 2D measurements. In turn, this model can be used in the context of a modified particle filter to predict future target locations. In the case where the target is occluded to some of the cameras, the missing measurements can be estimated using the facts that they must lie both in the subspace spanned by previous measurements and satisfy epipolar constraints. Hence, by exploiting both dynamical and geometrical constraints the proposed method can robustly handle substantial occlusion, without the need for performing 3D reconstruction, calibrated cameras or constraints on sensor separation. The performance of the proposed tracker is illustrated with several challenging examples involving targets that substantially change appearance and motion models while occluded to some of the cameras.

Fast algorithms for structured robust principal component analysis

A large number of problems arising in computer vision can be reduced to the problem of minimizing the nuclear norm of a matrix, subject to additional structural and sparsity constraints on its elements. Examples of relevant applications include, among others, robust tracking in the presence of outliers, manifold embedding, event detection, in-painting and tracklet matching across occlusion. In principle, these problems can be reduced to a convex semi-definite optimization form and solved using interior point methods. However, the poor scaling properties of these methods limit the use of this approach to relatively small sized problems. The main result of this paper shows that structured nuclear norm minimization problems can be efficiently solved by using an iterative Augmented Lagrangian Type (ALM) method that only requires performing at each iteration a combination of matrix thresholding and matrix inversion steps. As we illustrate in the paper with several examples, the proposed algorithm results in a substantial reduction of computational time and memory requirements when compared against interior-point methods, opening up the possibility of solving realistic, large sized problems.

An algorithm for fast constrained nuclear norm minimization and applications to systems identification

This paper presents a novel algorithm for efficiently minimizing the nuclear norm of a matrix subject to structural and semi-definite constraints. It requires performing only thresholding and eigenvalue decomposition steps and converges Q-superlinearly to the optimum. Thus, this algorithm offers substantial advantages, both in terms of memory requirements and computational time over conventional semi-definite programming solvers. These advantages are illustrated using as an example the problem of finding the lowest order system that interpolates a collection of noisy measurements.

Fast Structured Nuclear Norm Minimization With Applications to Set Membership Systems Identification

Set membership identification seeks to obtain models amenable to be used in a robust control framework. While under suitable assumptions this problem is convex, existing methods lead to high order models. As we show in this note, this difficulty can be avoided by recasting the problem as a structured nuclear norm minimization. To solve this problem, we propose a computationally efficient first order algorithm that requires performing only a combination of thresholding and eigenvalue decomposition steps. Finally, since the optimization is carried out only over sequences compatible with the output responses of stable systems, the identified model is guaranteed to be stable.

Blind identification of sparse dynamic networks and applications

This paper considers the problem of identifying the topology of a sparsely interconnected network of dynamical systems from experimental noisy data. Specifically, we assume that the observed data was generated by an underlying, unknown graph topology where each node corresponds to a given time-series and each link to an unknown autoregressive model that maps those time series. The goal is to recover the sparsest (in the sense of having the fewest number of links) structure compatible with some a-priori information and capable of explaining the observed data. Contrary to related existing work, our framework allows for (unmeasurable) exogenous inputs, intended to model relatively infrequent events such as environmental or set-point changes in the underlying processes. The main result of the paper shows that both the network topology and the unknown inputs can be identified by solving a convex optimization problem, obtained by combining Group-Lasso type arguments with a re-weighted heuristics. As shown here, this combination leads to substantially sparser topologies than using either group Lasso or orthogonal decomposition based algorithms. These results are illustrated using both academic examples and several non-trivial problems drawn from multiple application domains that include finances, biology and computer vision.

Finding Causal Interactions in Video Sequences

This paper considers the problem of detecting causal interactions in video clips. Specifically, the goal is to detect whether the actions of a given target can be explained in terms of the past actions of a collection of other agents. We propose to solve this problem by recasting it into a directed graph topology identification, where each node corresponds to the observed motion of a given target, and each link indicates the presence of a causal correlation. As shown in the paper, this leads to a block-sparsification problem that can be efficiently solved using a modified Group-Lasso type approach, capable of handling missing data and outliers (due for instance to occlusion and mis-identified correspondences). Moreover, this approach also identifies time instants where the interactions between agents change, thus providing event detection capabilities. These results are illustrated with several examples involving non-trivial interactions amongst several human subjects.

Convex relaxations for robust identification of Wiener systems and applications

This paper considers the identification of Wiener systems in a worst case framework. Given some a priori information about the admissible set of plants, nonlinearities and measurement noise, and a posteriori experimental data, our goal is twofold: (i) establish whether the a priori and a posteriori information are consistent, and (ii) in that case find a model that interpolates the available experimental information within the noise level. As recently shown, this problem is generically NP hard both in the number of data points and the number of inputs to the non-linearity. Our main result shows that a computationally attractive relaxation can be obtained by recasting the problem as a rank-constrained semi-definite optimization and using existing tools specifically tailored to this type of problems. These results are illustrated with a practical application drawn from computer vision

Euclidean structure recovery from motion in perspective image sequences via Hankel rank minimization

In this paper we consider the problem of recovering 3D Euclidean structure from multi-frame point correspondence data in image sequences under perspective projection. Existing approaches rely either only on geometrical constraints reflecting the rigid nature of the object, or exploit temporal information by recasting the problem into a nonlinear filtering form. In contrast, here we introduce a new constraint that implicitly exploits the temporal ordering of the frames, leading to a provably correct algorithm to find Euclidean structure (up to a single scaling factor) without the need to alternate between projective depth and motion estimation, estimate the Fundamental matrices or assume a camera motion model. Finally, the proposed approach does not require an accurate calibration of the camera. The accuracy of the algorithm is illustrated using several examples involving both synthetic and real data.

Using dynamics to recover Euclidian 3-dimensional structure from 2-dimensional perspective projections

In this paper we consider the problem of recovering the 3-dimensional Euclidian structure of a rigid object from multi-frame point correspondence data in a sequence of 2-D images obtained under perspective projection. The main idea is to recast the problem as the identification of an LTI system based on partial data. The main result of the paper shows that, under mild conditions, the lowest order system whose projections interpolate the 2-D data, yields (up to a single scaling constant) the correct 3 dimensional Euclidean coordinates of the points. Finally, we show that the problem of finding this system (and hence the associated 3-D data) can be recast into a rank minimization form that can be efficiently solved using convex relaxations. In contrast, existing approaches to the problem, based on iterative matrix factorizations can recover structure only up to a projective transformation that does not preserve the Euclidian geometry of the object.