Euncheol Choi

Super‐resolution approach to overcome physical limitations of imaging sensors: An overview

Although the performance of CCD and CMOS imaging sensors has improved since their invention, they still have several physical limitations, such as various sources of noise, limited dynamic range, and limited spatial resolution. Besides these physical limitations, they have malfunctioning problems, such as smearing and blooming, which degrade the quality of captured images. These limitations and malfunctioning problems can be overcome, based on device physics and circuit technology. However, a signal‐processing‐based approach is a good alternative solution to these problems, because it may cost less and existing imaging systems can be still utilized. In a broad sense, this signal‐processing‐based approach can be called a super‐resolution approach. The goal of this article is to introduce a super‐resolution approach that overcomes the limitations of imaging sensors. To this purpose, we describe the existing limitations of imaging sensors first, and then describe the corresponding super‐resolution approach.

Smear removal algorithm using the optical black region for CCD imaging sensors

Smearing is an inevitable phenomenon that occurs during the charge transfer process with CCD imaging sensors. For still images, smearing can be reduced by using a mechanical shutter or special drive methods, but these techniques cannot be applied to image sequences. In this paper, we propose a smear removal algorithm that can be applied to imaging systems for not only still images but also image sequences. The proposed algorithm uses the optical black region (OBR), a group of pixels located in the boundary of CCD imaging sensors. Although the OBR is not generally exposed to light, it contains smear information caused by the charge transport. For this reason, the proposed method estimates the smear component from the signal recorded in the OBR and produces a corrected OBR signal by removing noise. Then, the corrected OBR signal is uniformly subtracted from the signals in the effective pixel region to eliminate the smear effect. Moreover, the saturated pixels are substituted by the weighted average of the neighboring pixels to prevent visual degradation.

Object tracking based on area weighted centroids shifting with spatiality constraints

Recently, kernel-based tracking algorithms such as the mean shift tracking algorithm has been proposed, which use the information of color histogram together with some spatial information provided by the kernel. However, in spite of the fast speed, there exists an inherent instability problem which is due to the use of an isotropic kernel for spatiality and the use of the Bhattacharyya coefficient as the similarity function. In this paper, we will analyze how the use of the kernel and the Bhattacharyya coefficient can arouse the instability problem. Based on the analysis, we propose a tracking scheme that uses a new representation of the location of the target which is constrained by the color, the area, and the spatiality information of the target in a more stable way than the mean shift algorithm. With this representation, the target localization in the next frame can be achieved by a direct one step computation, and the tracking becomes stable, even in difficult situations such as low-rate-frame environment, and partial occlusion.

Striping noise removal of satellite images by nonlinear mapping

The striping noise removal method of an along-track scanned satellite image is considered in this paper. Nonuniformity of detectors caused by imperfect calibration and the drift of detector characteristics generates striping noise. The proposed nonlinear mapping consists of offset component correction (OCC) and nonlinear component correction (NCC). OCC is executed first under the assumption that the tendency of temporal (column) mean changes slowly across the detectors. Secondly, NCC, which is the least square approach for each of the same input intensity, is performed to reflect the nonlinear characteristics of the detector. The effectiveness of the proposed algorithm is demonstrated experimentally with real satellite images.

Dimension Reduction and Pre-emphasis for Compression of Hyperspectral Images

As the dimensionality of remotely sensed data increases, the need for efficient compression algorithms for hyperspectral images also increases. However, when hyperspectral images are compressed with conventional image compression algorithms, which have been developed to minimize mean squared errors, discriminant information necessary to distinguish among classes may be lost during compression process. In this paper, we propose to enhance such discriminant information prior to compression. In particular, we first find a new basis where class separability is better represented by applying a feature extraction method. However, due to high correlations between adjacent bands of hyperspectral data, we have singularity problems in applying feature extraction methods. In order to address the problem, we first reduce the dimension of data and then find a new basis by applying a feature extraction algorithm. Finally, dominant discriminant features are enhanced and the enhanced data are compressed using a conventional compression algorithm such as 3D SPIHT. Experiments show that the proposed compression method provides improved classification accuracies compared to the existing compression algorithms.

Scale-adaptive object tracking using color centroids

We propose a stable scale-adaptive tracking method that uses the centroids of the target colors in the target localization and scale adaptation. Because of the spatial information inherent in the centroids, a direct relationship can be established between the centroids and the scale of the target region. After the zooming factors are calculated, the unreliable zooming factors are filtered out to produce a reliable zooming factor that determines the new scale of the target.

Object Tracking Algorithm Using Weighted Color Centroids Shifting

Recently, mean shift tracking algorithms have been proposed which use the information of color histogram together with some spatial information provided by the kernel. In spite of their fast speed, the algorithms are suffer from an inherent instability problem which is due to the use of an isotropic kernel for spatiality and the use of the Bhattacharyya coefficient as a similarity function. In this paper, we analyze how the kernel and the Bhattacharyya coefficient can arouse the instability problem. Based on the analysis, we propose a novel tracking scheme that uses a new representation of the location of the target which is constrained by the color, the area, and the spatiality information of the target in a more stable way than the mean shift algorithm. With this representation, the target localization in the next frame can be achieved by one step computation, which makes the tracking stable, even in difficult situations such as low-rate-frame environment, and partial occlusion.

Object Tracking Based on Color Centroids Shifting with Background Color and Temporal filtering

With the development of mobile devices and intelligent surveillance system loaded with pan/tilt cameras, object tracking with non-stationary cameras has become a topic with increasing importancy. Since it is difficult to model a background image in a non-stationary camera environment, colors and texture are the most important features in the tracking algorithm. However, colors in the background similar to those in the target arise instability in the tracking. Recently, we proposed a robust color based tracking algorithm that uses an area weighted centroid shift. In this letter, we update the model such that it becomes more stable against background colors. The proposed algorithm also incorporates time filtering by adding an additional energy term to the energy functional.