Yanpeng Cao

Single-image-based solution for optics temperature-dependent nonuniformity correction in an uncooled long-wave infrared camera

In this Letter, we propose an efficient and accurate solution to remove temperature-dependent nonuniformity effects introduced by the imaging optics. This single-image-based approach computes optics-related fixed pattern noise (FPN) by fitting the derivatives of correction model to the gradient components, locally computed on an infrared image. A modified bilateral filtering algorithm is applied to local pixel output variations, so that the refined gradients are most likely caused by the nonuniformity associated with optics. The estimated bias field is subtracted from the raw infrared imagery to compensate the intensity variations caused by optics. The proposed method is fundamentally different from the existing nonuniformity correction (NUC) techniques developed for focal plane arrays (FPAs) and provides an essential image processing functionality to achieve completely shutterless NUC for uncooled long-wave infrared (LWIR) imaging systems.

Viewpoint invariant features from single images using 3D geometry

In this paper we present a novel approach for generating viewpoint invariant features from single images and demonstrate their application for robust matching over widely separated views. The key idea consists of retrieving building structure from single images and then utlising the recovered 3D geometry to improve the performances of feature extraction and matching. Urban environments usually contain many structured regularities, so that the images of those environments contain straight parallel lines and vanishing points, which can be efficiently exploited for 3D reconstruction. We present an effective scheme to recover 3D planar surfaces using the extracted line segments and their associated vanishing points. The viewpoint invariant features are then computed on the normalized front-parallel views of the obtained 3D planes. The advantages of the proposed approach include: (1) the new feature is very robust against perspective distortions and viewpoint changes due to its consideration of 3D geometry; (2) the features are completely computed from single images and do not need information from additional devices (e.g. stereo cameras, or active ranging devices). Experiments are carried out to demonstrate the proposed scheme ability to effectively handle very difficult wide baseline matching tasks in the presence of repetitive building structures and significant viewpoint changes.

Shutterless solution for simultaneous focal plane array temperature estimation and nonuniformity correction in uncooled long-wave infrared camera

In uncooled long-wave infrared (LWIR) microbolometer imaging systems, temperature fluctuations of the focal plane array (FPA) result in thermal drift and spatial nonuniformity. In this paper, we present a novel approach based on single-image processing to simultaneously estimate temperature variances of FPAs and compensate the resulting temperature-dependent nonuniformity. Through well-controlled thermal calibrations, empirical behavioral models are derived to characterize the relationship between the responses of microbolometer and FPA temperature variations. Then, under the assumption that strong dependency exists between spatially adjacent pixels, we estimate the optimal FPA temperature so as to minimize the global intensity variance across the entire thermal infrared image. We make use of the estimated FPA temperature to infer an appropriate nonuniformity correction (NUC) profile. The performance and robustness of the proposed temperature-adaptive NUC method are evaluated on realistic IR images obtained by a 640 × 512 pixels uncooled LWIR microbolometer imaging system operating in a significantly changed temperature environment.

Effective Strip Noise Removal for Low-Textured Infrared Images Based on 1-D Guided Filtering

Infrared images typically contain obvious strip noise. It is a challenging task to eliminate such noise without blurring fine image details in low-textured infrared images. In this paper, we introduce an effective single-image-based algorithm to accurately remove strip-type noise present in infrared images without causing blurring effects. First, a 1-D row guided filter is applied to perform edge-preserving image smoothing in the horizontal direction. The extracted high-frequency image part contains both strip noise and a significant amount of image details. Through a thermal calibration experiment, we discover that a local linear relationship exists between infrared data and strip noise of pixels within a column. Based on the derived strip noise behavioral model, strip noise components are accurately decomposed from the extracted high-frequency signals by applying a 1-D column guided filter. Finally, the estimated noise terms are subtracted from the raw infrared images to remove strips without blurring image details. The performance of the proposed technique is thoroughly investigated and is compared with the state-of-the-art 1-D and 2-D denoising algorithms using captured infrared images.

Robust alignment of wide baseline terrestrial laser scans via 3D viewpoint normalization

The complexity of natural scenes and the amount of information acquired by terrestrial laser scanners turn the registration among scans into a complex problem. This problem becomes even more challenging when two individual scans captured at significantly changed viewpoints (wide baseline). Since laser-scanning instruments nowadays are often equipped with an additional image sensor, it stands to reason making use of the image content to improve the registration process of 3D scanning data. In this paper, we present a novel improvement to the existing feature techniques to enable automatic alignment between two widely separated 3D scans. The key idea consists of extracting dominant planar structures from 3D point clouds and then utilizing the recovered 3D geometry to improve the performance of 2D image feature extraction and matching. The resulting features are very discriminative and robust to perspective distortions and viewpoint changes due to exploiting the underlying 3D structure. Using this novel viewpoint invariant feature, the corresponding 3D points are automatically linked in terms of wide baseline image matching. Initial experiments with real data demonstrate the potential of the proposed method for the challenging wide baseline 3D scanning data alignment tasks.

Robust wide baseline scene alignment based on 3D viewpoint normalization

This paper presents a novel scheme for automatically aligning two widely separated 3D scenes via the use of viewpoint invariant features. The key idea of the proposed method is following. First, a number of dominant planes are extracted in the SfM 3D point cloud using a novel method integrating RANSAC and MDL to describe the underlying 3D geometry in urban settings. With respect to the extracted 3D planes, the original camera viewing directions are rectified to form the front-parallel views of the scene. Viewpoint invariant features are extracted on the canonical views to provide a basis for further matching. Compared to the conventional 2D feature detectors (e.g. SIFT, MSER), the resulting features have following advantages: (1) they are very discriminative and robust to perspective distortions and viewpoint changes due to exploiting scene structure; (2) the features contain useful local patch information which allow for efficient feature matching. Using the novel viewpoint invariant features, wide-baseline 3D scenes are automatically aligned in terms of robust image matching. The performance of the proposed method is comprehensively evaluated in our experiments. Its demonstrated that 2D image feature matching can be significantly improved by considering 3D scene structure.

A multi-scale non-uniformity correction method based on wavelet decomposition and guided filtering for uncooled long wave infrared camera

Abstract In uncooled long-wave infrared (LWIR) imaging systems, non-uniformity of the amplifier in readout circuit will generate significant noise in captured infrared images. This type of noise, if not eliminated, may manifest as vertical and horizontal strips in the raw image and human observers are particularly sensitive to these types of image artifacts. In this paper we propose an effective non-uniformity correction (NUC) method to remove strip noise without loss of fine image details. This multi-scale destriping method consists of two consecutive steps. Firstly, wavelet-based image decomposition is applied to separate the original input image into three individual scale levels: large, median and small scales. In each scale level, the extracted vertical image component contains strip noise and vertical-orientated image textures. Secondly, a novel multi-scale 1D guided filter is proposed to further separate strip noise from image textures in each individual scale level. More specifically, in the small scale level, we choose a small filtering window for guided filter to eliminate strip noise. On the contrary, a large filtering window is used to better preserve image details from blurring in large scale level. Our proposed algorithm is systematically evaluated using real-captured infrared images and the quantitative comparison results with the state-of-the-art destriping algorithms demonstrate that our proposed method can better remove the strip noise without blurring image fine details.

Solid state temperature-dependent NUC (non-uniformity correction) in uncooled LWIR (long-wave infrared) imaging system

In uncooled LWIR microbolometer imaging systems, temperature fluctuations of FPA (Focal Plane Array) as well as lens and mechanical components placed along the optical path result in thermal drift and spatial non-uniformity. These non-idealities generate undesirable FPN (Fixed-Pattern-Noise) that is difficult to remove using traditional, individual shutterless and TEC-less (Thermo-Electric Cooling) techniques. In this paper we introduce a novel single-image based processing approach that marries the benefits of both statistical scene-based and calibration-based NUC algorithms, without relying neither on extra temperature reference nor accurate motion estimation, to compensate the resulting temperature-dependent non-uniformities. Our method includes two subsequent image processing steps. Firstly, an empirical behavioral model is derived by calibrations to characterize the spatio-temporal response of the microbolometric FPA to environmental and scene temperature fluctuations. Secondly, we experimentally establish that the FPN component caused by the optics creates a spatio-temporally continuous, low frequency, low-magnitude variation of the image intensity. We propose to make use of this property and learn a prior on the spatial distribution of natural image gradients to infer the correction function for the entire image. The performance and robustness of the proposed temperature-adaptive NUC method are demonstrated by showing results obtained from a 640×512 pixels uncooled LWIR microbolometer imaging system operating over a broad range of temperature and with rapid environmental temperature changes (i.e. from –5°C to 65°C within 10 minutes).

Single Image Augmented Reality Using Planar Structures in Urban Environments

In this paper, we present an effective method for integrating 3D augmented reality graphics into single images taken in urban environments. Building facades usually contain a large number of parallel lines aligned along several principal directions. We make use of the images of these 3D parallel lines and their corresponding vanishing points to recover a number of 3D planes from single 2D images of urban environments and then use them to represent the spatial layout of a scene. 3D objects are then aligned with respect to these recovered planes to achieve realistic augmented reality effects. In experiments we applied the proposed method to implement augmented reality in images from a benchmark image dataset of urban environments.

Spatially Adaptive Column Fixed-Pattern Noise Correction in Infrared Imaging System Using 1D Horizontal Differential Statistics

In this paper we present a novel non-uniformity correction (NUC) method to remove column fixed-pattern noise (FPN), which is introduced by non-uniformity of on-chip column-parallel readout circuit in uncooled infrared focal plane array. We first define a new image statistic measurement, which is named as 1D horizontal differential statistics, to differentiate column FPN from structural edges, and further propose a filtering scheme to adaptively compute noise terms in structure and non-structure regions by applying different correction models. The proposed NUC technique combines the advantages of global- and local-based correction methods, thus can effectively eliminate column FPN without losing original thermal details. The performance of the proposed method is systematically evaluated, and is compared with the state-of-the-art column FPN correction solutions using realistic infrared images.