Hassan Foroosh

Extension of phase correlation to subpixel registration

In this paper, we have derived analytic expressions for the phase correlation of downsampled images. We have shown that for downsampled images the signal power in the phase correlation is not concentrated in a single peak, but rather in several coherent peaks mostly adjacent to each other. These coherent peaks correspond to the polyphase transform of a filtered unit impulse centered at the point of registration. The analytic results provide a closed-form solution to subpixel translation estimation, and are used for detailed error analysis. Excellent results have been obtained for subpixel translation estimation of images of different nature and across different spectral bands.

Sparse Convolutional Neural Networks

Deep neural networks have achieved remarkable performance in both image classification and object detection problems, at the cost of a large number of parameters and computational complexity. In this work, we show how to reduce the redundancy in these parameters using a sparse decomposition. Maximum sparsity is obtained by exploiting both inter-channel and intra-channel redundancy, with a fine-tuning step that minimize the recognition loss caused by maximizing sparsity. This procedure zeros out more than 90% of parameters, with a drop of accuracy that is less than 1% on the ILSVRC2012 dataset. We also propose an efficient sparse matrix multiplication algorithm on CPU for Sparse Convolutional Neural Networks (SCNN) models. Our CPU implementation demonstrates much higher efficiency than the off-the-shelf sparse matrix libraries, with a significant speedup realized over the original dense network. In addition, we apply the SCNN model to the object detection problem, in conjunction with a cascade model and sparse fully connected layers, to achieve significant speedups.

Optimal local shape description for rotationally non-symmetric optical surface design and analysis

A local optical surface representation as a sum of basis functions is proposed and implemented. Specifically, we investigate the use of linear combination of Gaussians. The proposed approach is a local descriptor of shape and we show how such surfaces are optimized to represent rotationally non-symmetric surfaces as well as rotationally symmetric surfaces. As an optical design example, a single surface off-axis mirror with multiple fields is optimized, analyzed, and compared to existing shape descriptors. For the specific case of the single surface off-axis magnifier with a 3 mm pupil, >15 mm eye relief, 24 degree diagonal full field of view, we found the linear combination of Gaussians surface to yield an 18.5% gain in the average MTF across 17 field points compared to a Zernike polynomial up to and including 10th order. The sum of local basis representation is not limited to circular apertures.

View-invariant action recognition using fundamental ratios

A moving plane observed by a fixed camera induces a fundamental matrix F across multiple frames, where the ratios among the elements in the upper left 2times2 submatrix are herein referred to as the Fundamental Ratios. We show that fundamental ratios are invariant to camera parameters, and hence can be used to identify similar plane motions from varying viewpoints. For action recognition, we decompose a body posture into a set of point triplets (planes). The similarity between two actions is then determined by the motion of point triplets and hence by their associated fundamental ratios, providing thus view-invariant recognition of actions. Results evaluated over 255 semi-synthetic video data with 100 independent trials at a wide range of noise levels, and also on 56 real videos of 8 different classes of actions, confirm that our method can recognize actions under substantial amount of noise, even when they have dynamic timeline maps, and the viewpoints and camera parameters are unknown and totally different.

Subpixel estimation of shifts directly in the Fourier domain

In this paper, we establish the exact relationship between the continuous and the discrete phase difference of two shifted images, and show that their discrete phase difference is a two-dimensional sawtooth signal. Subpixel registration can, thus, be performed directly in the Fourier domain by counting the number of cycles of the phase difference matrix along each frequency axis. The subpixel portion is given by the noninteger fraction of the last cycle along each axis. The problem is formulated as an overdetermined homogeneous quadratic cost function under rank constraint for the phase difference, and the shape constraint for the filter that computes the group delay. The optimal tradeoff for imposing the constraints is determined using the method of generalized cross validation. Also, in order to robustify the solution, we assume a mixture model of inlying and outlying estimated shifts and truncate our quadratic cost function using expectation maximization.

View-Invariant Action Recognition from Point Triplets

We propose a new view-invariant measure for action recognition. For this purpose, we introduce the idea that the motion of an articulated body can be decomposed into rigid motions of planes defined by triplets of body points. Using the fact that the homography induced by the motion of a triplet of body points in two identical pose transitions reduces to the special case of a homology, we use the equality of two of its eigenvalues as a measure of the similarity of the pose transitions between two subjects, observed by different perspective cameras and from different viewpoints. Experimental results show that our method can accurately identify human pose transitions and actions even when they include dynamic timeline maps, and are obtained from totally different viewpoints with different unknown camera parameters.

Video Completion for Perspective Camera Under Constrained Motion

This paper presents a novel technique to fill in missing background and moving foreground of a video captured by a static or moving camera. Different from previous efforts which are typically based on processing in the 3D data volume, we slice the volume along the motion manifold of the moving object, and therefore reduce the search space from 3D to 2D, while still preserve the spatial and temporal coherence. In addition to the computational efficiency, based on geometric video analysis, the proposed approach is also able to handle real videos under perspective distortion, as well as common camera motions, such as panning, tilting, and zooming. The experimental results demonstrate that our algorithm performs comparably to 3D search based methods, and however extends the current state-of-the-art repairing techniques to videos with projective effects, as well as illumination changes

Trajectory Rectification and Path Modeling for Video Surveillance

Path modeling for video surveillance is an active area of research. We address the issue of Euclidean path modeling in a single camera for activity monitoring in a multi- camera video surveillance system. The paper proposes (i) a novel linear solution to auto-calibrate any camera observing pedestrians and (ii) to use these calibrated cameras to detect unusual object behavior. During the unsupervised training phase, after auto-calibrating a camera and metric rectifying the input trajectories, the input sequences are registered to the satellite imagery and prototype path models are constructed. This allows us to estimate metric information directly from the video sequences. During the testing phase, using our simple yet efficient similarity measures, we seek a relation between the input trajectories derived from a sequence and the prototype path models. We test the proposed method on synthetic as well as on real-world pedestrian sequences.

Camera calibration and light source orientation from solar shadows

In this paper, we describe a method for recovering camera parameters from perspective views of daylight shadows in a scene, given only minimal geometric information determined from the images. This minimal information consists of two 3D stationary points and their cast shadows on the ground plane. We show that this information captured in two views is sufficient to determine the focal length, the aspect ratio, and the principal point of a pinhole camera with fixed intrinsic parameters. In addition, we are also able to compute the orientation of the light source. Our method is based on exploiting novel inter-image constraints on the image of the absolute conic and the physical properties of solar shadows. Compared to the traditional methods that require images of some precisely machined calibration patterns, our method uses cast shadows by the sun, which are common in natural environments, and requires no measurements of any distance or angle in the 3D world. To demonstrate the accuracy of the proposed algorithm and its utility, we present the results on both synthetic and real images, and apply the method to an image-based rendering problem.

Action recognition using rank-1 approximation of Joint Self-Similarity Volume

In this paper, we make three main contributions in the area of action recognition: (i) We introduce the concept of Joint Self-Similarity Volume (Joint SSV) for modeling dynamical systems, and show that by using a new optimized rank-1 tensor approximation of Joint SSV one can obtain compact low-dimensional descriptors that very accurately preserve the dynamics of the original system, e.g. an action video sequence; (ii) The descriptor vectors derived from the optimized rank-1 approximation make it possible to recognize actions without explicitly aligning the action sequences of varying speed of execution or different frame rates; (iii) The method is generic and can be applied using different low-level features such as silhouettes, histogram of oriented gradients, etc. Hence, it does not necessarily require explicit tracking of features in the space-time volume. Our experimental results on three public datasets demonstrate that our method produces remarkably good results and outperforms all baseline methods.