Wesam Sakla

Action classification in polarimetric infrared imagery via diffusion maps

This work explores the application of a nonlinear dimensionality reduction technique known as diffusion maps for performing action classification in polarimetric infrared video sequences. The diffusion maps algorithm has been used successfully in a variety of applications involving the extraction of low-dimensional embeddings from high-dimensional data. Our dataset is composed of eight subjects each performing three basic actions: walking, walking while carrying an object in one hand, and running. The actions were captured with a polarized microgrid sensor operating in the longwave portion of the electromagnetic (EM) spectrum with a temporal resolution of 24 Hz, yielding the Stokes traditional intensity (S0) and linearly polarized (S1, S2) components of data. Our work includes the use of diffusion maps as an unsupervised dimensionality reduction step prior to action classification with three conventional classifiers: the linear perceptron algorithm, the k nearest neighbors (KNN) algorithm, and the kernel-based support vector machine (SVM). We present classification results using both the low-dimensional principal components via PCA and the low-dimensional diffusion map embedding coordinates of the data for each class. Results indicate that the diffusion map lower-dimensional embeddings provide a salient feature space for action classification, yielding an increase of overall classification accuracy by ~40% compared to PCA. Additionally, we examine the utility that the polarimetric sensor may provide by concurrently performing these analyses in the polarimetric feature spaces.

Vehicle tracking through the exploitation of remote sensing and LWIR polarization science

Vehicle tracking is an integral component in layered sensing exploitation applications. The utilization of a combination of sensing modalities and processing techniques provides better insight about a situation than can be achieved with a single sensing modality. In this work, several robust features are explored for vehicle tracking using data captured in a remote sensing setting. A target area is surveyed by a sensor operating capturing polarization information in the longwave infrared (LWIR) band. We here extend our previous work ([1]) to experimental analysis of several feature sets including three classic features (Stokes images, DoLP, the Degree of Linear Polarization, and AoP, the Angle of Polarization) and several geometry inspired features.1

Non-parametric bounds on the nearest neighbor classification accuracy based on the Henze-Penrose metric

Analysis procedures for higher-dimensional data are generally computationally costly; thereby justifying the high research interest in the area. Entropy-based divergence measures have proven their effectiveness in many areas of computer vision and pattern recognition. However, the complexity of their implementation might be prohibitive in resource-limited applications, as they require estimates of probability densities which are very difficult to compute directly for high-dimensional data. In this paper, we investigate the usage of a non-parametric distribution-free metric, known as the Henze-Penrose test statistic, to estimate the divergence between different classes of vehicles. In this regard, we apply some common feature extraction techniques to further characterize the distributional separation relative to the original data. Moreover, we employ the Henze-Penrose metric to obtain bounds for the Nearest Neighbor (NN) classification accuracy. Simulation results demonstrate the effectiveness and the reliability of this metric in estimating the inter-class separability. In addition, the proposed bounds are exploited for selecting the least number of features that would retain sufficient discriminative information.

Gaussian Nonlinear Line Attractor for learning multidimensional data

The human brains ability to extract information from multidimensional data modeled by the Nonlinear Line Attractor (NLA), where nodes are connected by polynomial weight sets. Neuron connections in this architecture assumes complete connectivity with all other neurons, thus creating a huge web of connections. We envision that each neuron should be connected to a group of surrounding neurons with weighted connection strengths that reduces with proximity to the neuron. To develop the weighted NLA architecture, we use a Gaussian weighting strategy to model the proximity, which will also reduce the computation times significantly. Once all data has been trained in the NLA network, the weight set can be reduced using a locality preserving nonlinear dimensionality reduction technique. By reducing the weight sets using this technique, we can reduce the amount of outputs for recognition tasks. An appropriate distance measure can then be used for comparing testing data and the trained data when processed through the NLA architecture. It is observed that the proposed GNLA algorithm reduces training time significantly and is able to provide even better recognition using fewer dimensions than the original NLA algorithm. We have tested this algorithm and showed that it works well in different datasets, including the EO Synthetic Vehicle database and the Sheffield face database.

An empirical comparison of K-SVD and GMRA for dictionary learning

The topic of constructing data-dependent dictionaries, referred to as dictionary learning, has received considerable interest in the past decade. In this work, we compare the ability of two dictionary learning algorithms, K-SVD and geometric multi-resolution analysis (GMRA), to perform image reconstruction using a fixed number of coefficients. K-SVD is an algorithm originating from the compressive sensing community and relies on optimization techniques. GMRA is a multi-scale technique that is based on manifold approximation of highdimensional point clouds of data. The empirical results of this work using a synthetic dataset of images of vehicles with diversity in viewpoint and lighting show that the K-SVD algorithm exhibits better generalization reconstruction performance with respect to test images containing lighting diversity that were not present in the construction of the dictionary, while GMRA exhibits superior reconstruction on the training data.

Gaussian weighted neighborhood connectivity of nonlinear line attractor for learning complex manifolds

The human brain has the capability to process high quantities of data quickly for detection and recognition tasks. These tasks are made simpler by the understanding of data, which intentionally removes redundancies found in higher dimensional data and maps the data onto a lower dimensional space. The brain then encodes manifolds created in these spaces, which reveal a specific state of the system. We propose to use a recurrent neural network, the nonlinear line attractor (NLA) network, for the encoding of these manifolds as specific states, which will draw untrained data towards one of the specific states that the NLA network has encoded. We propose a Gaussian-weighted modular architecture for reducing the computational complexity of the conventional NLA network. The proposed architecture uses a neighborhood approach for establishing the interconnectivity of neurons to obtain the manifolds. The modified NLA network has been implemented and tested on the Electro-Optic Synthetic Vehicle Model Database created by the Air Force Research Laboratory (AFRL), which contains a vast array of high resolution imagery with several different lighting conditions and camera views. It is observed that the NLA network has the capability for representing high dimensional data for the recognition of the objects of interest through its new learning strategy. A nonlinear dimensionality reduction scheme based on singular value decomposition has found to be very effective in providing a low dimensional representation of the dataset. Application of the reduced dimensional space on the modified NLA algorithm would provide fast and more accurate recognition performance for real time applications.

Sparse representation for vehicle recognition

The Sparse Representation for Classification (SRC) algorithm has been demonstrated to be a state-of-the-art algorithm for facial recognition applications. Wright et al. demonstrate that under certain conditions, the SRC algorithm classification performance is agnostic to choice of linear feature space and highly resilient to image corruption. In this work, we examined the SRC algorithm performance on the vehicle recognition application, using images from the semi-synthetic vehicle database generated by the Air Force Research Laboratory. To represent modern operating conditions, vehicle images were corrupted with noise, blurring, and occlusion, with representation of varying pose and lighting conditions. Experiments suggest that linear feature space selection is important, particularly in the cases involving corrupted images. Overall, the SRC algorithm consistently outperforms a standard k nearest neighbor classifier on the vehicle recognition task.

An objective multi-sensor fusion metric for target detection

Target detection is limited based on a specific sensors capability; however, the combination of multiple sensors will improve the confidence of target detection. Confidence of detection, tracking and identifying a target in a multi-sensor environment depends on intrinsic and extrinsic sensor qualities, e.g. target geo-location registration, and environmental conditions 1. Determination of the optimal sensors and classification algorithms, required to assist in specific target detection, has largely been accomplished with empirical experimentation. Formulation of a multi-sensor effectiveness metric (MuSEM) for sensor combinations is presented in this paper. Leveraging one or a combination of sensors should provide a higher confidence of target classification. This metric incorporates the Dempster-Shafer Theory for decision analysis. MuSEM is defined for weakly labeled multimodal data and is modeled and trained with empirical fused sensor detections; this metric is compared to Boolean algebra algorithms from decision fusion research. Multiple sensor specific classifiers are compared and fused to characterize sensor detection models and the likelihood functions of the models. For area under the curve (AUC), MuSEM attained values as high as .97 with an average difference of 5.33% between Boolean fusion rules. Data was collected from the Air Force Research Lab’s Minor Area Motion Imagery (MAMI) project. This metric is efficient and effective, providing a confidence of target classification based on sensor combinations.

Skin-based hyperspectral dismount detection using sparse representation

This paper presents a sparsity-based dismount detection algorithm for hyperspectral skin signatures in conjunction with the sequential forward feature selection (SFFS) scheme. The proposed sparsity-based detection (SD) approach relies on the observation that spectral signatures belonging to the same class approximately lie in a low-dimensional subspace. An unknown test sample can be represented by only a few training samples in the structured dictionary, and the underlying sparse representation vector contains discriminative information for detection. The proposed algorithm is applicable to both spectrally pure as well as mixed pixels. Experimental results show that the SD approach outperforms classical hyper-spectral detection algorithms such as the adaptive coherence estimation (ACE) algorithm, orthogonal subspace projection (OSP), and the adaptive matched subspace detector (AMSD).

Unmixing hyperspectral skin data using non-negative matrix factorization

The ability to accurately detect a target of interest in a hyperspectral imagery (HSI) is largely dependent on the spatial and spectral resolution. While hyperspectral imaging provides high spectral resolution, the spatial resolution is mostly dependent on the optics and distance from the target. Many times the target of interest does not occupy a full pixel and thus is concealed within a pixel, i.e. the target signature is mixed with other constituent material signatures within the field of view of that pixel. Extraction of spectral signatures of constituent materials from a mixed pixel can assist in the detection of the target of interest. Hyperspectral unmixing is a process to identify the constituent materials and estimate the corresponding abundances from the mixture. In this paper, a framework based on non-negative matrix factorization (NMF) is presented, which is utilized to extract the spectral signature and fractional abundance of human skin in a scene. The NMF technique is employed in a supervised manner such that the spectral bases of each constituent are computed first, and then these bases are applied to the mixed pixel. Experiments using synthetic and real data demonstrate that the proposed algorithm provides an effective supervised technique for hyperspectral unmixing of skin signatures.