Amy E. Galbraith

Resolution enhancement of multilook imagery for the multispectral thermal imager

This paper studies the feasibility of enhancing the spatial resolution of multilook Multispectral Thermal Imager (MTI) imagery using an iterative resolution enhancement algorithm known as Projection Onto Convex Sets (POCS). A multiangle satellite image modeling tool is implemented, and simulated multilook MTI imagery is formed to test the resolution enhancement algorithm. Experiments are done to determine the optimal configuration and number of multiangle low-resolution images needed for a quantitative improvement in the spatial resolution of the high-resolution estimate. The issues of atmospheric path radiance and directional reflectance variations are explored to determine their effect on the resolution enhancement performance.

Modeling the MTI electro-optic system sensitivity and resolution

We present an analysis methodology that offers efficient characterization of the Multispectral Thermal Imager (MTI) electro-optic system response to a wide range of user-specified system parameters and spectral scenarios. This methodology combines physics-based modeling of the MTI hardware with MTI prelaunch characterization data. The resulting models enable the user to generate application-specific sensitivity and resolution studies of the MTI image capture process, and aid in the development of calibration procedures and retrieval algorithms for MTI. In addition to quantifying the MTI response, the methodology developed in this paper is sufficiently general to permit the prototyping and evaluation of a variety of multispectral electro-optic systems. Finally, an example utilizing nominal orbital parameters and targeted MODTRAN scenarios that exercise the various spectral band functions is provided.

Resampling methods for the MTI coregistration product

Accurate coregistration of images from the Multispectral Thermal Imager (MTI) is needed to properly align bands for spectral analysis and physical retrievals, such as water surface temperature, land-cover classification, or small target identification. After accounting for spacecraft motion, optical distortion, and geometrical perspective, the irregularly-spaced pixels in the images must be resampled to a common grid. What constitutes an optimal resampling depends, to some extent, on the needs of the user. A good resampling trades off radiometric fidelity, contrast preservation for small objects, and cartographic accuracy -- and achieves this compromise without unreasonable computational effort. The standard MTI coregistration product originally used a weighted-area approach to achieve this irregular resampling, which generally over-smoothes the imagery and reduces the contrast of small objects. Recently, other resampling methods have been implemented to improve the final coregistered image. These methods include nearest-neighbor resampling and a tunable, distance-weighted resampling. We will discuss the pros and cons of various resampling methods applied to MTI images, and show results of comparing the contrast of small objects before and after resampling.

LANL experience with coregistration of MTI imagery

The fifteen-channel Multispectral Thermal Imager (MTI) provides accurately calibrated satellite imagery for a variety of scientific and programmatic purposes. To be useful, the calibrated pixels from the individual detectors on the focal plane of this pushbroom sensor must be resampled to a regular grid corresponding to the observed scene on the ground. In the LEVEL1B_R_COREG product, it is required that the pixels from different spectral bands and from different sensor chip assemblies all be coregistered to the same grid. For the LEVEL1B_R_GEO product, it is further required that this grid be georeferenced to the Universal Transverse Mercator coordinate system. It is important that an accurate registration is achieved, because most of the higher level products (e.g. ground reflectance) are derived from these LEVEL1B_R products. Initially, a single direct georeferencing approach was pursued for performing the coregistration task. Although this continues to be the primary algorithm for our automated pipeline registration, we found it advantageous to pursue alternative approaches as well. This paper surveys these approaches, and offers lessons learned during the three years we have been addressing the coregistration requirements for MTI imagery at the Los Alamos National Laboratory (LANL).

MTI science, data products and ground data processing overview

The mission of the Multispectral Thermal Imager (MTI) satellite is to demonstrate the efficacy of highly accurate multispectral imaging for passive characterization of urban and industrial areas, as well as sites of environmental interest. The satellite makes top-of-atmosphere radiance measurements that are subsequently processed into estimates of surface properties such as vegetation health, temperatures, material composition and others. The MTI satellite also provides simultaneous data for atmospheric characterization at high spatial resolution. To utilize these data the MTI science program has several coordinated components, including modeling, comprehensive ground-truth measurements, image acquisition planning, data processing and data interpretation and analysis. Algorithms have been developed to retrieve a multitude of physical quantities and these algorithms are integrated in a processing pipeline architecture that emphasizes automation, flexibility and programmability. In addition, the MTI science team has produced detailed site, system and atmospheric models to aid in system design and data analysis. This paper provides an overview of the MTI research objectives, data products and ground data processing.

Evolving feature extraction algorithms for hyperspectral and fused imagery

Hyperspectral imagery with moderate spatial resolution (/spl sim/30 m) presents an interesting challenge to feature extraction algorithm developers, as both spatial and spectral signatures may be required to identify the feature of interest. We describe a genetic programming software system, called GENIE, which augments the human scientist/analyst by evolving customized spatio-spectral feature extraction pipelines from training data provided via an intuitive, point-and-click interface. We describe recent work exploring geospatial feature extraction from hyperspectral imagery, and from a multi-instrument fused dataset. For hyperspectral imagery, we demonstrate our system on NASA Earth Observer 1 (EO-1) Hyperion imagery, applied to agricultural crop detection. We present an evolved pipeline, and discuss its operation. We also discuss work with multi-spectral imagery (DOE/NNSA Multispectral Thermal Imager) fused with USGS digital elevation model (DEM) data, with the application of detecting mixed conifer forest.

Remote Placement of Magnetically Coupled Ultrasonic Sensors for Structural Health Monitoring

In this work we develop an intelligent remote sensor placement system for standoff deployment of magnetically coupled ultrasonic sensors for structural health monitoring applications. Currently there exists significant legacy infrastructure that requires monitoring. Sensors often need to be accurately placed in hard-to-reach locations which are exposed to harsh environmental conditions, all while ensuring adequate mechanical coupling between the sensor and the structure. Installing these sensors is a task which is time consuming, expensive, and dangerous. In this paper, we develop an intelligent pneumatic remote sensor placement system meant to be integrated with commercially available multicopters. It is designed to accurately deploy sensor nodes from a standoff distance. To achieve this it will calculate the required trajectory and energy requirements to ensure proper placemen as well as coupling between the node and the structure without damaging the sensor package or the structure in the process. This work leverages recent advances in computer vision and commercially available multicopters to align the remote sensor placement system with the point of attachment on the structure. This technology will reduce the barriers associated with the deployment of large scale sensor networks in the field of structural health monitoring.

Simulating vision through time: Hierarchical, sparse models of visual cortex for motion imagery

Efficient pattern recognition in motion imagery has become a growing challenge as the number of video sources proliferates worldwide. Historically, automated analysis of motion imagery, such as object detection, classification and tracking, has been accomplished using hand-designed feature detectors. Though useful, these feature detectors are not easily extended to new data sets or new target categories since they are often task specific, and typically require substantial effort to design. Rather than hand-designing filters, recent advances in the field of image processing have resulted in a theoretical framework of sparse, hierarchical, learned representations that can describe video data of natural scenes at many spatial and temporal scales and many levels of object complexity. These sparse, hierarchical models learn the information content of imagery and video from the data itself and lead to state-of-the-art performance and more efficient processing. Processing efficiency is important as it allows scaling up of research to work with dataset sizes and numbers of categories approaching real-world conditions. We now describe recent work at Los Alamos National Laboratory developing hierarchical sparse learning computer vision models that can process high definition color video in real time. We present preliminary results extending our prior work on object classification in still imagery [1] to discovery of useful features at different time scales in motion imagery for detection, classification and tracking of objects.

LANL MTI science team experience

The Multispectral Thermal Imager (MTI) is a technology test and demonstration satellite whose primary mission involved a finite number of technical objectives. MTI was not designed, or supported, to become a general purpose operational satellite. The role of the MTI science team is to provide a core group of system-expert scientists who perform the scientific development and technical evaluations needed to meet programmatic objectives. Another mission for the team is to develop algorithms to provide atmospheric compensation and quantitative retrieval of surface parameters to a relatively small community of MTI users. Finally, the science team responds and adjusts to unanticipated events in the life of the satellite. Broad or general lessons learned include the value of working closely with the people who perform the calibration of the data as well as those providing archived image and retrieval products. Close interaction between the Los Alamos National Laboratory (LANL) teams was very beneficial to the overall effort as well as the science effort. Secondly, as time goes on we make increasing use of gridded global atmospheric data sets which are products of global weather model data assimilation schemes. The Global Data Assimilation System information is available globally every six hours and the Rapid Update Cycle products are available over much of the North America and its coastal regions every hour. Additionally, we did not anticipate the quantity of validation data or time needed for thorough algorithm validation. Original validation plans called for a small number of intensive validation campaigns soon after launch. One or two intense validation campaigns are needed but are not sufficient to define performance over a range of conditions or for diagnosis of deviations between ground and satellite products. It took more than a year to accumulate a good set of validation data. With regard to the specific programmatic objectives, we feel that we can do a reasonable job on retrieving surface water temperatures well within the 1°C objective under good observing conditions. Before the loss of the onboard calibration system, sea surface retrievals were usually within 0.5°C. After that, the retrievals are usually within 0.8°C during the day and 0.5°C at night. Daytime atmospheric water vapor retrievals have a scatter that was anticipated: within 20%. However, there is error in using the Aerosol Robotic Network retrievals as validation data which may be due to some combination of calibration uncertainties, errors in the ground retrievals, the method of comparison, and incomplete physics. Calibration of top-of-atmosphere radiance measurements to surface reflectance has proven daunting. We are not alone here: it is a difficult problem to solve generally and the main issue is proper compensation for aerosol effects. Getting good reflectance validation data over a number of sites has proven difficult but, when assumptions are met, the algorithm usually performs quite well. Aerosol retrievals for off-nadir views seem to perform better than near-nadir views and the reason for this is under investigation. Land surface temperature retrieval and temperature-emissivity separations are difficult to perform accurately with multispectral sensors. An interactive cloud masking system was implemented for production use. Clouds are so spectrally and spatially variable that users are encouraged to carefully evaluate the delivered mask for their own needs. The same is true for the water mask. This mask is generated from a spectral index that works well for deep, clear water, but there is much variability in water spectral reflectance inland and along coasts. The value of the second-look maneuvers has not yet been fully or systematically evaluated. Early experiences indicated that the original intentions have marginal value for MTI objectives, but potentially important new ideas have been developed. Image registration (the alignment of data from different focal planes) and band-to-band registration has been a difficult problem to solve, at least for mass production of the images in a processing pipeline. The problems, and their solutions, are described in another paper.

Evolving spatio-spectral feature extraction algorithms for hyperspectral imagery

Hyperspectral imagery data sets present an interesting challenge to feature extraction algorithm developers. Beyond the immediate problem of dealing with the sheer amount of spectral information per pixel in a hyperspectral image, the remote sensing scientist must explore a complex algorithm space in which both spatial and spectral signatures may be required to identify a feature of interest. Rather than carry out this algorithm exploration by hand, we are interested in developing learning systems that can evolve these algorithms. We describe a genetic programming/supervised classifier software system, called GENIE, which evolves image processing tools for remotely sensed imagery. Our primary application has been land-cover classification from satellite imagery. GENIE was developed to evolve classification algorithms for multispectral imagery, and the extension to hyperspectral imagery presents a chance to test a genetic programming system by greatly increasing the complexity of the data under analysis, as well as a chance to find interesting spatio-spectral algorithms for hyperspectral imagery. We demonstrate our system on publicly available imagery from the new Hyperion imaging spectrometer onboard the NASA Earth Observing-1 (EO-1) satellite.