Karmon Vongsy

The SHARE 2012 data campaign

A multi-modal (hyperspectral, multispectral, and LIDAR) imaging data collection campaign was conducted just south of Rochester New York in Avon, NY on September 20, 2012 by the Rochester Institute of Technology (RIT) in conjunction with SpecTIR, LLC, the Air Force Research Lab (AFRL), the Naval Research Lab (NRL), United Technologies Aerospace Systems (UTAS) and MITRE. The campaign was a follow on from the SpecTIR Hyperspectral Airborne Rochester Experiment (SHARE) from 2010. Data was collected in support of the eleven simultaneous experiments described here. The airborne imagery was collected over four different sites with hyperspectral, multispectral, and LIDAR sensors. The sites for data collection included Avon, NY, Conesus Lake, Hemlock Lake and forest, and a nearby quarry. Experiments included topics such as target unmixing, subpixel detection, material identification, impacts of illumination on materials, forest health, and in-water target detection. An extensive ground truthing effort was conducted in addition to collection of the airborne imagery. The ultimate goal of the data collection campaign is to provide the remote sensing community with a shareable resource to support future research. This paper details the experiments conducted and the data that was collected during this campaign.

Extension of the Linear Chromodynamics Model for Spectral Change Detection in the Presence of Residual Spatial Misregistration

A generalized likelihood ratio test (GLRT) statistic for spectral change detection based on the linear chromodynamics model is extended to accommodate unknown residual misregistration between imagery described by a prior probability density function for the spatial misregistration. Using a normal prior distribution leads to a fourth-order polynomial that can be numerically minimized over the unknown misregistration parameters. A more computationally efficient closed-form solution is developed based on a quadratic approximation and provides comparable results to the numerical minimization for the investigated test cases while running 30 times faster. The results applying the method to hyperspectral imagery indicate up to an order of magnitude reduction in false alarms at the same detection rate relative to baseline change detection methods for synthetically misregistered test data particularly in image regions containing edges and fine spatial features. Sensitivity to model parameters is assessed, and the method is compared with a previously published misregistration compensation approach yielding comparable results. Although the GLRT approach appears to exhibit comparable change detection performance, it offers the possibility of tailoring the algorithm to a priori knowledge of expected misregistration errors or to compensate structured misregistration as would occur due to parallax errors due to perspective variations (e.g., image parallax).

Engine classification using vibrations measured by Laser Doppler Vibrometer on different surfaces

In our previous studies, vehicle surfaces’ vibrations caused by operating engines measured by Laser Doppler Vibrometer (LDV) have been effectively exploited in order to classify vehicles of different types, e.g., vans, 2-door sedans, 4-door sedans, trucks, and buses, as well as different types of engines, such as Inline-four engines, V-6 engines, 1-axle diesel engines, and 2-axle diesel engines. The results are achieved by employing methods based on an array of machine learning classifiers such as AdaBoost, random forests, neural network, and support vector machines. To achieve effective classification performance, we seek to find a more reliable approach to pick authentic vibrations of vehicle engines from a trustworthy surface. Compared with vibrations directly taken from the uncooperative vehicle surfaces that are rigidly connected to the engines, these vibrations are much weaker in magnitudes. In this work we conducted a systematic study on different types of objects. We tested different types of engines ranging from electric shavers, electric fans, and coffee machines among different surfaces such as a white board, cement wall, and steel case to investigate the characteristics of the LDV signals of these surfaces, in both the time and spectral domains. Preliminary results in engine classification using several machine learning algorithms point to the right direction on the choice of type of object surfaces to be planted for LDV measurements.

Parallax mitigation for hyperspectral change detection

A pixel-level Generalized Likelihood Ratio Test (GLRT) statistic for hyperspectral change detection is developed to mitigate false change caused by image parallax. Change detection, in general, represents the difficult problem of discriminating significant changes opposed to insignificant changes caused by radiometric calibration, image registration issues, and varying view geometries. We assume that the images have been registered, and each pixel pair provides a measurement from the same spatial region in the scene. Although advanced image registration methods exist that can reduce mis-registration to subpixel levels; residual spatial mis-registration can still be incorrectly detected as significant changes. Similarly, changes in sensor viewing geometry can lead to parallax error in an urban cluttered scene where height structures, such as buildings, appear to move. Our algorithm looks to the inherent relationship between the image views and the theory of stereo vision to perform parallax mitigation leading to a search result in the assumed parallax direction. Mitigation of the parallax-induced false alarms is demonstrated using hyperspectral data in the experimental analysis. The algorithm is examined and compared to the existing chronochrome anomalous change detection algorithm to assess performance.

Detection of spectrally varying BRDF materials in hyperspectral reflectance imagery

This study examines the influence of non-Lambertian reflectance effects on the detection of subpixel vehicles in hyperspectral imagery. Object-level BRDF spectral signatures for an olive green sedan were simulated using a fast, radiometrically accurate 3D rendering model, and the signatures were embedded as subpixel targets of low fractional fill into the HyMap hyperspectral image provided in the Rochester Institute of Technology Blind Test dataset. Detection algorithms based on the ACE detector were run on the scene. The results demonstrate a significant improvement in detection performance when including the targets BRDF variation in the detection scheme through either a subspace ACE or a Bayesian selection (multiple ACE detector) method.

Classification of uncooperative vehicles with sparse laser Doppler vibrometry measurements

Recently Laser Doppler Vibrometry (LDV) has been widely employed to achieve long-range sensing in military applications, due to its high spatial and spectral resolutions in vibration measurements that facilitates effective analysis using signal processing and machine learning techniques. Based on the collaboration of The City College of New York and the Air Force Research Laboratory in the last several years, we have developed a bank of algorithms to classify different types of vehicles, such as sedans, vans, pickups, motor-cycles and buses, and identify various kinds of engines, such as Inline-4, V6, 1- and 2-axle truck engines. Thanks to the similarities of the LDV signals to acoustic and other time-series signals, a large of body of existing approaches in literature has been employed, such as speech coding, time series representation, Fourier analysis, pyramid analysis, support vector machine, random forest, neural network, and deep learning algorithms. We have found that the classification results based on some of these methods are extremely promising. For instance, our vehicle engine classification algorithm based on the pyramid Fourier analysis of the engine vibration and fundamental frequencies of vehicle surfaces over the data collected by our LDV in the summer of 2014 have consistently attained 96% precision. In laboratory studies or well-controlled environments, a great array of high quality LDV measured points all over the vehicles are permitted by the vehicle owners, therefore extensive classifier training can be conducted to effectively capture the innate properties of surfaces in the space and spectral domains. However, in real contested environments, which are of utmost interest and practical importance to military applications, the uncooperative vehicles are either fast moving or purposively concealed and thus not many high quality LDV measurements can be made. In this work an intensive study is performed to compare the performance in vehicle classifications under the cooperative and uncooperative environments via LDV measurements based on a content-based indexing approach. The method uses an iterative Fourier analysis and an artificial feed-forward neural network. As our empirical studies have suggested, even in uncooperative and contested environments, with adequate training dataset for similar vehicles, our classification approach can still yield promising recognition rates.

Integrating spatial & spectral information for change detection in hyperspectral imagery

Change detection (CD) is an important topic in the remote sensing community. Although many CD works exist using spatial information or spectral information only, few works have incorporated both in the CD process. We propose a fused spatial-spectral feature vector for use in a maximum likelihood correlation coefficient (MLCC)-based change detector where the resulting test statistic provides the ability to label changes as departures or arrivals relative to the reference image. Results show that incorporating both spatial and spectral information has an advantage over either one independently. Additionally, incorporating spatial and spectral information in the CD process adds some robustness in the presence of misregistration errors.

Toward prediction of hyperspectral target detection performance after lossy image compression

Hyperspectral imagery (HSI) offers numerous advantages over traditional sensing modalities with its high spectral content that allows for classification, anomaly detection, target discrimination, and change detection. However, this imaging modality produces a huge amount of data, which requires transmission, processing, and storage resources; hyperspectral compression is a viable solution to these challenges. It is well known that lossy compression of hyperspectral imagery can impact hyperspectral target detection. Here we examine lossy compressed hyperspectral imagery from data-centric and target-centric perspectives. The compression ratio (CR), root mean square error (RMSE), the signal to noise ratio (SNR), and the correlation coefficient are computed directly from the imagery and provide insight to how the imagery has been affected by the lossy compression process. With targets present in the imagery, we perform target detection with the spectral angle mapper (SAM) and adaptive coherence estimator (ACE) and evaluate the change in target detection performance by examining receiver operating characteristic (ROC) curves and the target signal-to-clutter ratio (SCR). Finally, we observe relationships between the data- and target-centric metrics for selected visible/near-infrared to shortwave infrared (VNIR/SWIR) HSI data, targets, and backgrounds that motivate potential prediction of change in target detection performance as a function of compression ratio.

Effects of fundamental frequency normalization on vibration-based vehicle classification

Vibrometry offers the potential to classify a target based on its vibration spectrum. Signal processing is necessary for extracting features from the sensing signal for classification. This paper investigates the effects of fundamental frequency normalization on the end-to-end classification process [1]. Using the fundamental frequency, assumed to be the engine’s firing frequency, has previously been used successfully to classify vehicles [2, 3]. The fundamental frequency attempts to remove the vibration variations due to the engine’s revolution per minute (rpm) changes. Vibration signatures with and without fundamental frequency are converted to ten features that are classified and compared. To evaluate the classification performance confusion matrices are constructed and analyzed. A statistical analysis of the features is also performed to determine how the fundamental frequency normalization affects the features. These methods were studied on three datasets including three military vehicles and six civilian vehicles. Accelerometer data from each of these data collections is tested with and without normalization.

Remote vibrometry vehicle classification

In vehicle target classification, contact sensors have frequently been used to collect data to simulate laser vibrometry data. Accelerometer data has been used in numerous literature to test and train classifiers instead of laser vibrometry data [1] [2]. Understanding the key similarities and differences between accelerometer and laser vibrometry data is essential to keep progressing aided vehicle recognition systems. This paper investigates the contrast of accelerometer and laser vibrometer data on classification performance. Research was performed using the end-to-end process previously published by the authors to understand the effects of different types of data on the classification results. The end-to-end process includes preprocessing the data, extracting features from various signal processing literature, using feature selection to determine the most relevant features used in the process, and finally classifying and identifying the vehicles. Three data sets were analyzed, including one collection on military vehicles and two recent collections on civilian vehicles. Experiments demonstrated include: (1) training the classifiers using accelerometer data and testing on laser vibrometer data, (2) combining the data and classifying the vehicle, and (3) different repetitions of these tests with different vehicle states such as idle or revving and varying stationary revolutions per minute (rpm).