David P. Haefner

Human vision noise model validation for the U.S. Army sensor performance metric

Abstract. Image noise originating from a sensor system is often the limiting factor in target acquisition performance, especially when limited by atmospheric transmission or low-light conditions. To accurately predict target acquisition range performance for a wide variety of imaging systems, image degradation introduced by the sensor must be properly combined with the limitations of the human visual system (HVS). This crucial step of incorporating the HVS has been improved and updated within NVESD’s latest imaging system performance model. The new noise model discussed here shows how an imaging system’s noise and blur are combined with the contrast threshold function (CTF) to form the system CTF. Model calibration constants were found by presenting low-contrast sine gratings with additive noise in a two alternative forced choice experiment. One of the principal improvements comes from adding an eye photon noise term allowing the noise CTF to be accurate over a wide range of luminance. The latest HVS noise model is then applied to the targeting task performance metric responsible for predicting system performance from the system CTF. To validate this model, human target acquisition performance was measured from a series of infrared and visible-band noise-limited imaging systems.

Noise estimation of an MTF measurement

The modulation transfer function (MTF) measurement is critical for understanding the performance of an EOIR system. Unfortunately, due to both spatially correlated and spatially un-correlated noise sources, the performance of the MTF measurement (specifically near the cutoff) can be severely degraded. When using a 2D imaging system, the intrinsic sampling of the 1D edge spread function (ESF) allows for redundant samples to be averaged suppressing the noise contributions. The increase in the signal-to-noise will depend on the angle of the edge with respect to the sampling along with the specified re-sampling rate. In this paper, we demonstrate how the information in the final ESF can be used to identify the contribution of noise. With an estimate of the noise, the noise-limited portion of MTF measurement can be identified. Also, we demonstrate how the noise-limited portion of the MTF measurement can be used in combination with a fitting routine to provide a smoothed measurement.

Modeling boost performance using a two dimensional implementation of the targeting task performance metric

Using post-processing filters to enhance image detail, a process commonly referred to as boost, can significantly affect the performance of an EO/IR system. The US Armys target acquisition models currently use the Targeting Task Performance (TTP) metric to quantify sensor performance. The TTP metric accounts for each element in the system including: blur and noise introduced by the imager, any additional post-processing steps, and the effects of the Human Visual System (HVS). The current implementation of the TTP metric assumes spatial separability, which can introduce significant errors when the TTP is applied to systems using non-separable filters. To accurately apply the TTP metric to systems incorporating boost, we have implement a two-dimensional (2D) version of the TTP metric. The accuracy of the 2D TTP metric was verified through a series of perception experiments involving various levels of boost. The 2D TTP metric has been incorporated into the Night Vision Integrated Performance Model (NV-IPM) allowing accurate system modeling of non-separable image filters.

An imaging system detectivity metric using energy and power spectral densities

The purpose of this paper is to construct a robust modeling framework for imaging systems in order to predict the performance of detecting small targets such as Unmanned Aerial Vehicles (UAVs). The underlying principle is to track the flow of scene information and statistics, such as the energy spectra of the target and power spectra of the background, through any number of imaging components. This information is then used to calculate a detectivity metric. Each imaging component is treated as a single linear shift invariant (LSI) component with specified input and output parameters. A component based approach enables the inclusion of existing component-level models and makes it directly compatible with image modeling software such as the Night Vision Integrated Performance Model (NV-IPM). The modeling framework also includes a parallel implementation of Monte Carlo simulations designed to verify the analytic approach. However, the Monte Carlo simulations may also be used independently to accurately model nonlinear processes where the analytic approach fails, allowing for even greater extensibility. A simple trade study is conducted comparing the modeling framework to the simulation.

Comparing and contrasting different broadband MTF definitions and the relationship to range performance predictions

The monochromatic modulation transfer function (MTF) is a spectral average across wavelength weighted by the sensor’s spectral sensitivity and scaled by the spectral behavior of the source. For reflective band sensors, where there are significant variations in spectral shape of the reflected light, this spectral averaging can result in very different MTFs and, therefore, the resulting performance. In this paper, we explore the influence of this spectral averaging on performance utilizing NV-IPM v1.1 (Night Vision Integrated Performance Model). We report the errors in range performance when a system is characterized with one illumination and the performance is quoted for another. Our results summarize the accuracy of different assumptions to how a monochromatic MTF can be approximated, and how the measurement conditions under which a system was characterized should be considered when modeling performance.

NVLabCAP: an NVESD-developed software tool to determine EO system performance

Engineers at the US Army Night Vision and Electronic Sensors Directorate have recently developed a software package called NVLabCap. This software not only captures sequential frames from thermal and visible sensors, but it also can perform measurements of signal intensity transfer function, 3-dimensional noise, field of view, super-resolved modulation transfer function, and image bore sight. Additionally, this software package, along with a set of commonly known inputs for a given thermal imaging sensor, can be used to automatically create an NV-IPM element for that measured system. This model data can be used to determine if a sensor under test is within certain tolerances, and this model can be used to objectively quantify measured versus given system performance.

Thermal system field performance predictions from laboratory and field measurements

Laboratory measurements on thermal imaging systems are critical to understanding their performance in a field environment. However, it is rarely a straightforward process to directly inject thermal measurements into thermal performance modeling software to acquire meaningful results. Some of the sources of discrepancies between laboratory and field measurements are sensor gain and level, dynamic range, sensor display and display brightness, and the environment where the sensor is operating. If measurements for the aforementioned parameters could be performed, a more accurate description of sensor performance in a particular environment is possible. This research will also include the procedure for turning both laboratory and field measurements into a system model.

Signal intensity transfer function determination on thermal systems with stray light or scattering present

Accurate Signal Intensity Transfer Functions (SITF) measurements are necessary to determine the calibration factor in the 3D noise calculation of an electro-optical imaging system. The typical means for measuring a sensor’s SITF is to place the sensor in a flooded field environment at a distance that is relatively close to the aperture of the emitter. Unfortunately, this arrangement has the potential to allow for additional contributions to the SITF in the form of scattering or stray light if the optics are not designed properly in the system under test. Engineers at the US Army Night Vision and Electronic Sensors Directorate are working to determine a means of evaluating the contribution due to scatting or stray light.

Color camera measurement and modeling for use in the Night Vision Integrated Performance Model (NV-IPM)

The necessity of color balancing in day color cameras complicates both laboratory measurements as well as modeling for task performance prediction. In this proceeding, we discuss how the raw camera performance can be measured and characterized. We further demonstrate how these measurements can be modeled in the Night Vision Integrated Performance Model (NV-IPM) and how the modeled results can be applied to additional experimental conditions beyond those used during characterization. We also present the theoretical framework behind the color camera component in NV-IPM, where an effective monochromatic imaging system is created from applying a color correction to the raw color camera and generating the color corrected grayscale image. The modeled performance shows excellent agreement with measurements for both monochromatic and colored scenes. The NV-IPM components developed for this work are available in NV-IPM v1.2.

Performance assessment of compressive sensing imaging

Compressive sensing (CS) can potentially form an image of equivalent quality to a large format, megapixel array, using a smaller number of individual measurements. This has the potential to provide smaller, cheaper, and lower bandwidth imaging systems. To properly assess the value of such systems, it is necessary to fully characterize the image quality, including artifacts, sensitivity to noise, and CS limitations. Full resolution imagery of an eight tracked vehicle target set at range was used as an input for simulated single-pixel CS camera measurements. The CS algorithm then reconstructs images from the simulated single-pixel CS camera for various levels of compression and noise. For comparison, a traditional camera was also simulated setting the number of pixels equal to the number of CS measurements in each case. Human perception experiments were performed to determine the identification performance within the trade space. The performance of the nonlinear CS camera was modeled with the Night Vision Integrated Performance Model (NVIPM) by mapping the nonlinear degradations to an equivalent linear shift invariant model. Finally, the limitations of compressive sensing modeling will be discussed.