Testing and performance standards for elevated skin temperature (EST) screening systems using infrared cameras

Abstract

The COVID-19 pandemic has caused a renewed interest in Elevated Skin Temperature (EST) screening systems using infrared cameras. A standard for infrared EST system performance exists (IEC 80601-2-59) but few commercially available systems meet all the performance requirements of this standard. FDA researchers [Ghassemi et al.], in 2018, tested two commercially available EST systems to the IEC standard. They concluded the systems largely met the IEC standards but identified several ambiguities in the standard and, for some tests, proposed alternate test methods and criteria. Later they used these two systems in a large-scale clinical study testing the effectiveness for fever screening by comparing to oral temperature measurements [Zhou et al.]. In this paper we review the Ghassemi IEC test results and build on their work by recommending even further changes and clarifications to the IEC standard. Rationale for the recommended changes is illustrated with example test data collected using a prototype EST system. We also present a detailed treatment of how to do a complete system uncertainty analysis for these systems, identifying some inconsistencies in earlier treatments. The IEC standard includes requirements for accuracy, drift, stability, image uniformity, MRTD, spatial resolution and system uncertainty analysis. Ghassemi proposed an alternate test method and criteria for uniformity based on an “allpixel standard deviation” approach. We tested a prototype EST system to the IEC 80601-2-59 standard and with the alternate tests suggested by Ghassemi. We review pros, cons and limitations of each method and show how a 3D system noise approach (2D spatial plus temporal) can be used to mitigate limitations of both prior methods. From our results and review of Ghassemi we conclude that one of the two systems tested by Ghassemi likely did not meet the IEC specifications as presently written. Considering the very similar (and favorable) clinical results found by Zhou when they tested the two Ghassemi systems, we conclude the IEC standard as written is too stringent in some areas. In other respects, we found it has gaps and ambiguities. Some of these likely arose from attempts to address early interpreted-EST systems that relied on an operator to interpret the thermal image of the subject and make a subjective fever/no-fever determination. In hopes of improving the IEC standard, we recommend several changes.