Xinqiao Liu

Modeling and estimation of FPN components in CMOS image sensors

Fixed pattern noise (FPN) for a CCD sensor is modeled as a sample of a spatial white noise process. This model is, however, not adequate for characterizing FPN in CMOS sensors, since the redout circuitry of CMOS sensors and CCDs are very different. The paper presents a model for CMOS FPN as the sum of two components: a column and a pixel component. Each component is modeled by a first order isotropic autoregressive random process, and each component. Each component is modeled by a first order isotropic autoregressive random process, and each component is assumed to be uncorrelated with the other. The parameters of the processes characterize each component of the FPN and the correlations between neighboring pixels and neighboring columns for a batch of sensor. We show how to estimate the model parameters from a set of measurements, and report estimates for 64 X 64 passive pixel sensor (PPS) and active pixel sensor (APS) test structures implemented in a 0.35 micron CMOS process. High spatial correlations between pixel components were measured for the PPS structures, and between the column components in both PPS and APS. The APS pixel components were uncorrelated.

Simultaneous image formation and motion blur restoration via multiple capture

Advances in CMOS image sensors enable fast image capture, which makes it possible to capture multiple images within a normal exposure time. An algorithm that takes advantage of this capability by simultaneously constructing a high dynamic range image and performing motion blur restoration from multiple image captures is described. The algorithm comprises two main procedures-photocurrent estimation and motion/saturation detection. It operates completely locally-each pixels final value is computed using only its captured values-and recursively, requiring the storage of only a constant number of values per pixel independent of the number of images captured. These modest computational and storage requirements make it feasible to integrate all needed memory and processing with the image sensor on a single CMOS chip. Simulation results demonstrate the enhanced SNR, dynamic range, and the motion blur restoration obtained using our algorithm.

Synthesis of high dynamic range motion blur free image from multiple captures

Advances in CMOS image sensors enable high-speed image readout, which makes it possible to capture multiple images within a normal exposure time. Earlier work has demonstrated the use of this capability to enhance sensor dynamic range. This paper presents an algorithm for synthesizing a high dynamic range, motion blur free, still image from multiple captures. The algorithm consists of two main procedures, photocurrent estimation and saturation and motion detection. Estimation is used to reduce read noise, and, thus, to enhance dynamic range at the low illumination end. Saturation detection is used to enhance dynamic range at the high illumination end as previously proposed, while motion blur detection ensures that the estimation is not corrupted by motion. Motion blur detection also makes it possible to extend exposure time and to capture more images, which can be used to further enhance dynamic range at the low illumination end. Our algorithm operates completely locally; each pixels final value is computed using only its captured values, and recursively, requiring the storage of only a constant number of values per pixel independent of the number of images captured. Simulation and experimental results demonstrate the enhanced signal-to-noise ratio (SNR), dynamic range, and the motion blur prevention achieved using the algorithm.

Photocurrent estimation from multiple nondestructive samples in CMOS image sensor

CMOS image sensors generally suffer form lower dynamic range than CCDs due to their higher readout noise. Their high speed readout capability and the potential of integrating memory and signal processing with the sensor on the same chip, open up many possibilities for enhancing their dynamic range. Earlier work have demonstrated the use of multiple non-destructive samples to enhance dynamic range, while achieving higher SNR than using other dynamic range enhancement schemes. The high dynamic range image is constructed by appropriately scaling each pixels last sample before saturation. Conventional CDS is used to reduce offset FPN and reset noise. This simple high dynamic range image construction scheme, however, does not take full advantage of the multiple samples. Readout noise power, which doubles as a result of performing CDS, remain as high as in conventional sensor operation. As a result dynamic range is only extended at the high illumination end. The paper explores the use of linear mean-square-error estimation to more fully exploit the multiple pixel samples to reduce readout noise and thus extend dynamic range at the low illumination end. We present three estimation algorithms: (1) a recursive estimator when reset noise and offset FPN are ignored, (2) a non-recursive algorithm when reset noise and FPN are considered, and (3) a recursive estimation algorithm for case (2), which achieves mean square error close to the non-recursive algorithm without the need to store all the samples. The later recursive algorithm is attractive since it requires the storage of only a few pixel values per pixel, which makes its implementation in a single chip digital imaging system feasible.

A 10 kframe/s 0.18 /spl mu/m CMOS digital pixel sensor with pixel-level memory

A 352/spl times/288 pixel CMOS image sensor with pixel-level single-slope ADC and 8 b 3T DRAM cells achieves 9.4/spl times/9.4 /spl mu/m/sup 2/ pixel in standard 0.18 /spl mu/m CMOS. Continuous 10 kframes/s (1 Gpixels/s) 8 b per pixel snapshot image acquisition is achieved with 0.1% rms temporal noise and 0.18% rms FPN.

Active pixel sensors fabricated in a standard 0.18-μm CMOS technology

CMOS image sensors have benefitted from technology scaling down to 0.35 micrometers with only minor process modifications. Several studies have predicted that below 0.25 micrometers , it will become difficult, if not impossible to implement CMOS image sensors with acceptable performance without more significant process modifications. To explore the imaging performance of CMOS Image sensors fabricated in standard 0.18 micrometers technology, we designed a set of single pixel photodiode and photogate APS test structures. The test structures include pixels with different size n+/pwell and nwell/psub photodiodes and nMOS photogates. To reduce the leakages due to the in-pixel transistors, the follower, photogate, and transfer devices all use 3.3V thick oxide transistors. The paper reports on the key imaging parameters measured from these test structures including conversion gain, dark current and spectral response. We find that dark current density decreases super-linearly in reverse bias voltage, which suggest that it is desirable to run the photodetectors at low bias voltages. We find that QE is quite low due to high pwell doping concentration. Finally we find that the photogate circuit suffered from high transfer gate off current. QE is not significantly affected by this problem, however.

Experimental high speed CMOS image sensor system and applications

CMOS image sensors are capable of very high-speed non-destructive readout, enabling many novel applications. To explore such applications, we designed and prototyped an experimental high speed imaging system based on a CMOS digital pixel sensor (DPS). The experimental system comprises a PCB that has the DPS chip interfaced to a PC via three I/O cards supported by an easy to use software environment. The system is capable of image acquisition at rates of up to 1,400 frames/sec. After describing the DPS chip and experimental imaging system,we present two applications: dynamic range extension and optical flow estimation. These applications rely on the DPSs ability to perform non-destructive readout of multiple frames at high-speed.

Characterization of CMOS image sensors with Nyquist rate pixel-level ADC

Techniques for characterizing CCD imagers have been developed over many years. These techniques have been recently modified and extended to CMOS PPS and APS imagers. With the scaling of CMOS technology, an increasing number of transistors can be added to each pixel. A promising direction to utilize these transistors is to perform pixel level ADC. The authors have designed and prototyped two imagers with pixel level Nyquist rate ADC. The ADCs operate in parallel and output data one bit at a time. The data is read out of the imager array one bit plane at a time in a manner similar to a digital memory. Existing characterization techniques could not be directly used for these imagers, however, since there is no facility to read out the analog pixel values before ADC, and the ADC resolution is limited to only 8 bits. Fortunately, the ADCs are fully testable electrically without the need for any light or optics. This makes it possible obtain the ADC transfer curve, which greatly simplifies characterization. In this paper we describe how we characterize our pixel level ADC imagers. To estimate QE, we measure the imager photon to DN transfer curve and the ADC transfer curve. We find that both curves are quite linear.Using an estimate of the sense node capacitance we then estimate sensitivity, and QE. To estimate FPN we model it as an outcome of the sum of two uncorrelated random processes, one representing the ADC FPN, and the other representing the photodetector FPN, and develop estimators for the model parameters form imager data under uniform illumination. We report characterization result for a 640 by 512 imager, which was fabricated in a 0.35 micrometers standard digital CMOS process.