Kang-Yu Ni

Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisition

An aerial multiple camera tracking paradigm needs to not only spot unknown targets and track them, but also needs to know how to handle target reacquisition as well as target handoff to other cameras in the operating theater. Here we discuss such a system which is designed to spot unknown targets, track them, segment the useful features and then create a signature fingerprint for the object so that it can be reacquired or handed off to another camera. The tracking system spots unknown objects by subtracting background motion from observed motion allowing it to find targets in motion, even if the camera platform itself is moving. The area of motion is then matched to segmented regions returned by the EDISON mean shift segmentation tool. Whole segments which have common motion and which are contiguous to each other are grouped into a master object. Once master objects are formed, we have a tight bound on which to extract features for the purpose of forming a fingerprint. This is done using color and simple entropy features. These can be placed into a myriad of different fingerprints. To keep data transmission and storage size low for camera handoff of targets, we try several different simple techniques. These include Histogram, Spatiogram and Single Gaussian Model. These are tested by simulating a very large number of target losses in six videos over an interval of 1000 frames each from the DARPA VIVID video set. Since the fingerprints are very simple, they are not expected to be valid for long periods of time. As such, we test the shelf life of fingerprints. This is how long a fingerprint is good for when stored away between target appearances. Shelf life gives us a second metric of goodness and tells us if a fingerprint method has better accuracy over longer periods. In videos which contain multiple vehicle occlusions and vehicles of highly similar appearance we obtain a reacquisition rate for automobiles of over 80% using the simple single Gaussian model compared with the null hypothesis of <20%. Additionally, the performance for fingerprints stays well above the null hypothesis for as much as 800 frames. Thus, a simple and highly compact single Gaussian model is useful for target reacquisition. Since the model is agnostic to view point and object size, it is expected to perform as well on a test of target handoff. Since some of the performance degradation is due to problems with the initial target acquisition and tracking, the simple Gaussian model may perform even better with an improved initial acquisition technique. Also, since the model makes no assumption about the object to be tracked, it should be possible to use it to fingerprint a multitude of objects, not just cars. Further accuracy may be obtained by creating manifolds of objects from multiple samples.

Manifold-based fingerprinting for target identification

In this paper, we propose a fingerprint analysis algorithm based on using product manifolds to create robust signatures for individual targets in motion imagery. The purpose of target fingerprinting is to reidentify a target after it disappears and then reappears due to occlusions or out of camera view and to track targets persistently under camera handoff situations. The proposed method is statistics-based and has the benefit of being compact and invariant to viewpoint, rotation, and scaling. Moreover, it is a general framework and does not assume a particular type of objects to be identified. For improved robustness, we also propose a method to detect outliers of a statistical manifold formed from the training data of individual targets. Our experiments show that the proposed framework is more accurate in target reidentification than single-instance signatures and patch-based methods.

SAR moving target imaging using sparse and low-rank decomposition

We propose a method to image a complex scene with spotlight synthetic aperture radar (SAR) despite the presence of multiple moving targets. Many recent methods use sparsity-based reconstruction coupled with phase error corrections of moving targets to reconstruct stationary scenes. However, these methods rely on the assumption that the scene itself is sparse and thus unfortunately cannot handle realistic SAR scenarios with complex backgrounds consisting of more than just a few point targets. Our method makes use of sparse and low-rank (SLR) matrix decomposition, an efficient method for decomposing a low-rank matrix and sparse matrix from their sum. For detecting the moving targets and reconstructing the stationary background, SLR uses a convex optimization model that penalizes the nuclear norm of the low rank background structure and the L1 norm of the sparse moving targets. We propose an L1-norm regularization reconstruction method to form the input data matrix, which is grossly corrupted by the moving targets. Each column of the input matrix is a reconstructed SAR image with measurements from a small number of azimuth angles. The use of the L1-norm regularization and a sparse transform permits us to reconstruct the scene with significantly fewer measurements so that moving targets are approximately stationary. We demonstrate our SLR-based approach using simulations adapted from the GOTCHA Volumetric SAR data set. These simulations show that SLR can accurately image multiple moving targets with different individual motions in complex scenes where methods that assume a sparse scene would fail.

Accurate reconstruction of frequency-sparse signals from non-uniform samples

With the advent of a new sampling theory in recent years, compressed sensing (CS), it is possible to reconstruct signals from measurements far below the Nyquist rate. The CS theory assumes that signals are sparse and that measurement matrices satisfy certain conditions. Even though there have been many promising results, unfortunately there still exists a gap between the theory and actual real world applications. This is because of the fundamental problem that the CS formulation is discrete. We propose a sampling and reconstructing method for frequency-sparse signals that addresses this issue. The signals in our scenario are supported in a continuous sparsifying domain rather than discrete. This work focuses on a typical case in which the unknowns are frequencies and amplitudes. However, directly looking for the unknowns that best fit the measurements in the least-squares sense is a non-convex optimization problem, because sinusoids are oscillatory. Our approach extends the utility of CS to simplify this problem to a locally convex problem, hence making the solutions tractable. Direct measurements are taken from non-uniform time-samples, which is an extension of the CS problem with a subsampled Fourier matrix. The proposed reconstruction algorithm iteratively approximates the solutions using CS and then accurately solves for the frequencies with Newtons method and for the amplitudes with linear least squares solutions. Our simulations show success in accurate reconstruction of signals with arbitrary frequencies and significantly outperform current spectral compressed sensing methods in terms of reconstruction fidelity for both noise-free and noisy cases.

Feature transformation of neural activity with sparse and low-rank decomposition

We propose a novel application of the sparse and low-rank (SLR) decomposition method to decode cognitive states for concept activity measured using fMRI BOLD. Current decoding methods attempt to reduce the dimensionality of fMRI BOLD signals to increase classification rate, but do not address the separable issues of multiple noise sources and complexity in the underlying data. Our feature transformation method extends SLR to separate task activity from the resting state and extract concept specific cognitive state. We show a significant increase in single trial decoding of concepts from fMRI BOLD using SLR to extract task specific cognitive state.

Context and Task-Aware Knowledge Enhanced Compressive Imaging

We describe a foveated compressive sensing approach for image analysis applications that utilizes knowledge of the task to be performed to reduce the number of required measurements compared to conventional Nyquist sampling and compressive sensing based approaches. Our Compressive Optical Foveated Architecture (COFA) adapts the dictionary and compressive measurements to structure and sparsity in the signal, task, and scene by reducing measurement and dictionary mutual coherence and increasing sparsity using principles of actionable information and foveated compressive sensing. Actionable information is used to extract task-relevant regions of interest (ROIs) from a low-resolution scene analysis by eliminating the effects of nuisances for occlusion and anomalous motion detection. From the extracted ROIs, preferential measurements are taken using foveation as part of the compressive sensing adaptation process. The task-specific measurement matrix is optimized by using a novel saliency-weighted coherence minimization with respect to the learned signal dictionary. This incorporates the relative usage of the atoms in the dictionary. Therefore, the measurement matrix is not random, as in conventional compressive sensing, but is based on the dictionary structure and atom distributions. We utilize a patch-based method to learn the signal priors. A treestructured dictionary of image patches using KSVD is learned which can sparsely represent any given image patch with the tree-structure. We have implemented COFA in an end-to-end simulation of a vehicle fingerprinting task for aerial surveillance using foveated compressive measurements adapted to hierarchical ROIs consisting of background, roads, and vehicles. Our results show 113x reduction in measurements over conventional sensing and 28x reduction over compressive sensing using random measurements.

L1-methods for low-power surveillance

In this paper we introduce two novel methods for application of `1-minimization. In the first method, sparse and low-rank decomposition and compressive sensing-based retrieval are combined and applied to a low power surveillance model. The method exploits the ability of sparse and low-rank decompositions to extract significant and stationary features and the ability of compressive sensing approaches to reduce the number of measurements necessary. In the second method, a contiguity prior is added to compressive sensing methods on images and a numerical approach is proposed to solve this novel problem. Results are displayed in which the contiguity constrained method is applied to the low power surveillance model.