Francine K. Amon

Evaluation of thermal imaging cameras used in fire fighting applications

Thermal imaging cameras are rapidly becoming integral equipment for first responders for use in structure fires. Currently there are no standardized test methods or performance metrics available to the users or manufacturers of these instruments. The Building and Fire Research Laboratory (BFRL) at the National Institute of Standards and Technology (NIST) is developing a testing facility and methods to evaluate the performance of thermal imagers used by fire fighters to search for victims and hot spots in burning structures. The facility will test the performance of currently available imagers and advanced fire detection systems, as well as serve as a test bed for new technology. An evaluation of the performance of different thermal imaging detector technologies under field conditions is also underway. Results of this project will provide a quantifiable physical and scientific basis upon which industry standards for imaging performance, testing protocols and reporting practices related to the performance of thermal imaging cameras can be developed. The background and approach that shape the evaluation procedure for the thermal imagers are the primary focus of this paper.

Development of a performance evaluation facility for fire fighting thermal imagers

The Building and Fire Research Laboratory (BFRL) at the National Institute of Standards and Technology (NIST) is developing a new bench-scale testing facility and methods to evaluate the performance of thermal imagers used by fire fighters to search for victims and hot spots in burning structures. A larger-scale laboratory testing facility was constructed in 2002. This facility was used to determine the effects of water sprays on the imaging performance of a selection of thermal imagers. A new, smaller-scale laboratory facility, currently under construction, will provide a carefully controlled laboratory setting in which aspects of the environment inside a burning structure are simulated as closely as possible. It will also serve as a test bed for new technology. An evaluation of the performance of different thermal imaging detector technologies under field conditions is also underway. Results of this project will provide a quantifiable physical and scientific basis upon which industry standards for imaging performance, testing protocols and reporting practices related to the performance of thermal imaging cameras can be developed. In this paper a description of the testing facilities, including both generations of laboratory apparatus is presented.

Application of spatial frequency response as a criterion for evaluating thermal imaging camera performance

Police, firefighters, and emergency medical personnel are examples of first responders that are utilizing thermal imaging cameras in a very practical way every day. However, few performance metrics have been developed to assist first responders in evaluating the performance of thermal imaging technology. This paper describes one possible metric for evaluating spatial resolution using an application of Spatial Frequency Response (SFR) calculations for thermal imaging. According to ISO 12233, the SFR is defined as the integrated area below the Modulation Transfer Function (MTF) curve derived from the discrete Fourier transform of a camera image representing a knife-edge target. This concept is modified slightly for use as a quantitative analysis of the cameras performance by integrating the area between the MTF curve and the cameras characteristic nonuniformity, or noise floor, determined at room temperature. The resulting value, which is termed the Effective SFR, can then be compared with a spatial resolution value obtained from human perception testing of task specific situations to determine the acceptability of the performance of thermal imaging cameras. The testing procedures described herein are being developed as part of a suite of tests for possible inclusion into a performance standard on thermal imaging cameras for first responders.

First responder thermal imaging cameras: establishment of representative performance testing conditions

Thermal imaging cameras are rapidly becoming integral equipment for first responders for use in structure fires and other emergencies. Currently there are no standardized performance metrics or test methods available to the users and manufacturers of these instruments. The Building and Fire Research Laboratory (BFRL) at the National Institute of Standards and Technology is conducting research to establish test conditions that best represent the environment in which these cameras are used. First responders may use thermal imagers for field operations ranging from fire attack and search/rescue in burning structures, to hot spot detection in overhaul activities, to detecting the location of hazardous materials. In order to develop standardized performance metrics and test methods that capture the harsh environment in which these cameras may be used, information has been collected from the literature, and from full-scale tests that have been conducted at BFRL. Initial experimental work has focused on temperature extremes and the presence of obscuring media such as smoke. In full-scale tests, thermal imagers viewed a target through smoke, dust, and steam, with and without flames in the field of view. The fuels tested were hydrocarbons (methanol, heptane, propylene, toluene), wood, upholstered cushions, and carpeting with padding. Gas temperatures, CO, CO2, and O2 volume fraction, emission spectra, and smoke concentrations were measured. Simple thermal bar targets and a heated mannequin fitted in firefighter gear were used as targets. The imagers were placed at three distances from the targets, ranging from 3 m to 12 m.

Measurement of effective temperature range of fire service thermal imaging cameras

The use of thermal imaging cameras (TIC) by the fire service is increasing as fire fighters become more aware of the value of these tools. The National Fire Protection Association (NFPA) is currently developing a consensus standard for design and performance requirements of TIC as used by the fire service. The National Institute of Standards and Technology facilitates this process by providing recommendations for science-based performance metrics and test methods to the NFPA technical committee charged with the development of this standard. A suite of imaging performance metrics and test methods, based on the harsh operating environment and limitations of use particular to the fire service, has been proposed for inclusion in the standard. The Effective Temperature Range (ETR) measures the range of temperatures that a TIC can view while still providing useful information to the user. Specifically, extreme heat in the field of view tends to inhibit a TICs ability to discern surfaces having intermediate temperatures, such as victims and fire fighters. The ETR measures the contrast of a target having alternating 25 °C and 30 °C bars while an increasing temperature range is imposed on other surfaces in the field of view. The ETR also indicates the thermal conditions that trigger a shift in integration time common to TIC employing microbolometer sensors. The reported values for this imaging performance metric are the hot surface temperature range within which the TIC provides adequate bar contrast, and the hot surface temperature at which the TIC shifts integration time.

Development of infrared scene projectors for testing fire-fighter cameras

We have developed two types of infrared scene projectors for hardware-in-the-loop testing of thermal imaging cameras such as those used by fire-fighters. In one, direct projection, images are projected directly into the camera. In the other, indirect projection, images are projected onto a diffuse screen, which is then viewed by the camera. Both projectors use a digital micromirror array as the spatial light modulator, in the form of a Micromirror Array Projection System (MAPS) engine having resolution of 800 x 600 with mirrors on a 17 micrometer pitch, aluminum-coated mirrors, and a ZnSe protective window. Fire-fighter cameras are often based upon uncooled microbolometer arrays and typically have resolutions of 320 x 240 or lower. For direct projection, we use an argon-arc source, which provides spectral radiance equivalent to a 10,000 Kelvin blackbody over the 7 micrometer to 14 micrometer wavelength range, to illuminate the micromirror array. For indirect projection, an expanded 4 watt CO2 laser beam at a wavelength of 10.6 micrometers illuminates the micromirror array and the scene formed by the first-order diffracted light from the array is projected onto a diffuse aluminum screen. In both projectors, a well-calibrated reference camera is used to provide non-uniformity correction and brightness calibration of the projected scenes, and the fire-fighter cameras alternately view the same scenes. In this paper, we compare the two methods for this application and report on our quantitative results. Indirect projection has an advantage of being able to more easily fill the wide field of view of the fire-fighter cameras, which typically is about 50 degrees. Direct projection more efficiently utilizes the available light, which will become important in emerging multispectral and hyperspectral applications.

Suite of proposed imaging performance metrics and test methods for fire service thermal imaging cameras

The use of thermal imaging cameras (TIC) by the fire service is increasing as fire fighters become more aware of the value of these tools. The National Fire Protection Association (NFPA) is currently developing a consensus standard for design and performance requirements for TIC as used by the fire service. This standard will include performance requirements for TIC design robustness and image quality. The National Institute of Standards and Technology facilitates this process by providing recommendations for science-based performance metrics and test methods to the NFPA technical committee charged with the development of this standard. A suite of imaging performance metrics and test methods based on the harsh operating environment and limitations of use particular to the fire service has been proposed for inclusion in the standard. The performance metrics include large area contrast, effective temperature range, spatial resolution, nonuniformity, and thermal sensitivity. Test methods to measure TIC performance for these metrics are in various stages of development. An additional procedure, image recognition, has also been developed to facilitate the evaluation of TIC design robustness. The pass/fail criteria for each of these imaging performance metrics are derived from perception tests in which image contrast, brightness, noise, and spatial resolution are degraded to the point that users can no longer consistently perform tasks involving TIC due to poor image quality.

First responder thermal imaging cameras: development of performance metrics and test methods

Thermal imaging cameras are rapidly becoming integral equipment for first responders for use in structure fires and other emergencies. Currently, there are no standardized performance metrics or test methods available to the users and manufacturers of these instruments. The Building and Fire Research Laboratory at the National Institute of Standards and Technology is developing performance evaluation techniques that combine aspects of conventional metrics such as the contrast transfer function (CTF), the minimum resolvable temperature difference (MRTD), and noise equivalent temperature difference (NETD) with test methods that accommodate the special conditions in which first responders use these instruments. First responders typically use thermal imagers when their vision is obscured due to the presence of smoke, dust, fog, and/or the lack of visible light, and in cases when the ambient temperature is uncomfortably hot. Testing has shown that image contrast, as measured using a CTF calculation, suffers when a target is viewed through obscuring media. A proposed method of replacing the trained observer required for the conventional MRTD test method with a CTF calculation is presented. A performance metric that combines thermal resolution with target temperature and sensitivity mode shifts is also being investigated. Results of this work will support the establishment of standardized performance metrics and test methods for thermal imaging cameras that are meaningful to the first responders that use them.

LCD display screen performance testing for handheld thermal imaging cameras

Handheld thermal imaging cameras are an important tool for the first responder community. As their use becomes more prevalent, it will become important for a set of standard test metrics to be available to characterize the performance of these cameras. A major factor in the performance of the imagers is the quality of the image on a display screen. An imager may employ any type of display screen, but the results of this paper will focus on those using liquid crystal displays. First responders, especially firefighters, in the field rely on the performance of this screen to relay vital information during critical situations. Current research on thermal imaging camera performance metrics for first responder applications uses trained observer tests or camera composite output signal measurements. Trained observer tests are subjective and composite output tests do not evaluate the performance of the complete imaging system. It is the goal of this work to develop a non-nondestructive, objective method that tests the performance of the entire thermal imaging camera system, from the infrared destructive, sensor to the display screen. Application of existing display screen performance metrics to thermal imaging cameras requires additional consideration. Most display screen test metrics require a well defined electronic input, with either full black or white pixel input, often encompassing detailed spatial patterns and resolution. Well characterized thermal inputs must be used to obtain accurate, repeatable, and non-destructive display screen measurements for infrared cameras. For this work, a thermal target is used to correlate the measured camera output with the actual display luminance. A test method was developed to determine display screen luminance. A well characterized CCD camera and digital recording device were used to determine an electro-optical transfer function for thermal imaging cameras. This value directly relates the composite output signal to the luminance of the display screen, providing a realistic characterization of system performance.

Development of performance metrics and test methods for first responder location and tracking systems

The need for the development of performance requirements, metrics and test methods for evaluation of location and tracking systems is increasing as these systems begin to become commercially available for use by emergency responders. A working group was recently formed at the 5th Annual PPL Workshop consisting of members of the various emergency responder communities, representatives of government agencies, and location and tracking system manufacturers with plans to collect and share information about their ongoing efforts that have bearing on this effort. The goal of the working group is to develop baseline performance requirements for reliable and interoperable indoor location systems for emergency responders and develop test procedures to validate vendor systems against those requirements. The final deliverable will be a proposed draft document in acceptable format to be presented to appropriate standards developing organization(s). The following text describes in broad terms the steps by which the draft document will be created.