Fernanda D. V. R. Oliveira

CMOS Imager With Focal-Plane Analog Image Compression Combining DPCM and VQ

CMOS imagers, in comparison to CCD image sensors, have the great advantage of allowing for the implementation of signal processing circuitry inside the pixel matrix. We can extract information of interest from an image prior to analog-to-digital conversion. In this work, we present a 32 × 32 imaging integrated circuit that captures and compresses gray scale images on the focal plane of the image sensor using analog circuits that implement, for every 4 × 4 pixel block, differential pulse-code modulation, linear transform, and vector quantization. Theoretical details and circuit design are carefully described, as well as the test setup and details of the chip that was fabricated in a 0.35 μm CMOS technology. To validate the technique, we present tests and experimental results including overall modulation transfer function and photographs captured by the chip. The CMOS imager features focal-plane data compression based on DPCM and VQ with bit rate below 0.94 bpp and peak signal-to-noise ratio values around 18 dB. The overall power consumption is 37 mW (white image), which is equivalent to approximately 36 μW per pixel. Using photographs taken from bar-target pattern inputs, it is shown that details up to 2 cycles/cm are preserved in the decoded images.

Comparison of low-complexity image compression algorithms for analog circuit implementation

CMOS imaging hardware has become a widespread research topic due to its flexibility in allowing focal-plane, pixel-level signal processing. In a number of vision chips, dedicated image processing techniques are carried out in the analog domain to extract relevant information. Analog processing leads to time and power consumption savings, especially if combined with parallel processing, which is conventionally used because of the inherently parallel nature of imaging arrays. We focus on the focal-plane analog implementation of an image compression algorithm, based on differential pulse-code modulation, linear transform, and vector quantization. In a prototype fabricated in a 0.35 μm CMOS technology, the texture information inside each pixel block in a 32 × 32 array was locally encoded by inner products applied to 4 × 4 pixel neighborhoods followed by vector quantization. In this paper, a new chip implemented in a 0.18 μm CMOS technology incorporating improvements suggested by the previous prototype experimental evaluation is presented. We highlight a 5 dB increase in image quality, confirmed by numerical simulations, at the expense of modest increase in complexity and bit rate. Cascode current mirrors are used in parts of the circuitry and pixel non-linearity models are made available to the decoder.

A CMOS imaging system featuring focal-plane compression based on DPCM and VQ

We recently introduced a new focal plane image compression algorithm that is implemented with 607 transistors inside every 4 × 4 pixel block of a CMOS imager, using conventional 0.35 μm integration technology. This work focuses on preliminary results concerning the overall MTF of an imaging system in which the CMOS imager features focal-plane data compression based on DPCM and VQ with an overall bit rate below 0.94 bpp. Using bar-target pattern inputs, it is shown that details up to 2 cycles/cm are preserved in the decoded images.

A circuit for focal-plane image compression using vector quantization

Active pixel sensors are very suitable for the implementation of a large variety of image processing algorithms at the focal plane level. We propose a new focal plane image compression algorithm that is implemented with 607 transistors inside every 4×4 pixel block of a CMOS imager, using conventional 0.35 μm integration technology. We describe the theory of the proposed method, which is based on DPCM of the average block luminance and on VQ of four low-frequency components obtained by a linear transformation applied to the local pixels. We introduce the analog hardware of the pixel block and, to validate our design, present image coding results obtained from electrical Spice simulations of 64 pixel blocks.

Influence of cascode and simple current mirrors in inner product implementations for CMOS imagers

CMOS image sensors are suitable for smart camera designs because this technology allows the implementation of the pixel array and processing circuitry on the same chip. Due to this feature, CMOS imagers have been studied in industry and academy for various applications. This paper refers to an imager that performs data compression using analog current-mode hardware. The compression is based on linear transform, VQ and DPCM, and it was implemented in a previous imager fabricated with 0.35 μm CMOS technology. Experimental results obtained from this chip suggested some improvements for a new design, which was implemented with 0.18 μm CMOS technology. In this paper we highlight and justify the use of cascode current mirrors instead of simple current mirrors for the linear transform inner product operation. This modification leads to a significant increase in the image PSNR at the expense of a more complex layout. We also present a quadratic model used to reduce the distortion caused by the photodiode readout circuit.

Focal-Plane Scale Space Generation with a 6T Pixel Architecture

Aiming at designing a CMOS image sensor that combines high fill factor and focal-plane implementation of instrumental image processing steps, we propose a simple modification in a standard pixel architecture in order to allow for charge redistribution among neighboring pixels. As a result, averaging operations may be performed at the focal plane, and image smoothing based on Gaussian filtering may thus be implemented. By averaging neighboring pixel values, it is also possible to generate intermediate data structures that are required for the computation of Haar-like features. To show that the proposed hardware is suitable for computer vision applications, we present a systemlevel comparison in which the scale-invariant feature transform (SIFT) algorithm is executed twice: first, on data obtained with a classical Gaussian filtering approach, and then on data generated from the proposed approach. Preliminary schematic and extracted layout pixel simulations are also presented. Introduction Per-pixel pre-processing on spatial image sensor samples discards redundant data, thus decreasing the demands posed on the readout circuitry and enhancing SWaP (size, weight, and power) factors of camera systems based on these smart sensors. Image sensor architectures with embedded per-pixel processing have been proposed for a large variety of tasks [1], and their industrial exploitation is ramping up [2]. However, all these architectures share common drawbacks, namely increased pixel pitches, reduced fill factors, and lack of compatibility with advanced CIS technologies. All-in-all these drawbacks result in image quality degradation. Based on our previous results on image sensors with perpixel calculation of Gaussian filters, this paper presents a sixtransistor image pixel that is aimed at overcoming previous drawbacks, while performing Gaussian filtering and allowing the subsequent calculation of image key-points from the filter outputs [3]. As compared to previous sensors with embedded per-pixel Gaussian filtering [4][5], this 6T pixel, which occupies 6.28 × 6.28μm2 in a 110 nm CIS technology, yields 78% pitch reduction and 253% fill factor enlargement. With a two-transistor-per-pixel overhead with respect to conventional 4T pixels, the proposed image sensor architecture implements the following functions: • image capture, which is performed in the same way as in the 4T conventional architecture; • neighbor pixel value averaging, realized by charge redistribution; • image smoothing, by combining pixel values inside 2 × 2 pixel blocks and reconfiguring the array so that a Gaussian filter approximation is performed on an image having one quarter the resolution of the original image [6]; • scale-space [7] generation, by repeatedly applying the Gaussian filter. The proposed architecture performs simple parallel operations for the entire pixel matrix while images are captured. The processing capability of the proposed hardware can be used to optimize, in terms of processing time and power consumption, early vision tasks of image processing algorithms. The scale invariant feature transform (SIFT) [7], used for object recognition, and the Viola-Jones [8] object detector are examples of two algorithms that benefit from the presented pixel; in the first case with the scale-space generation and in the second by helping Haar-like feature computation. This paper addresses the SIFT algorithm and contextualizes the hardware-based solution in the algorithm processing flow. The pixel architecture is explained in the next section. Then, we show how it is possible to generate the scale-space for the SIFT with the proposed hardware. By the end of the paper, system level simulations are shown, as well as schematic and layout Spectre simulations. Proposed Pixel Architecture In the classical 4T architecture a pinned photodiode is connected to a floating diffusion through a transfer gate transistor. When the transfer gate is activated, all the charge stored by the photodiode, which is proportional to the incident light, is sent to the floating diffusion node where it is read and sent to the output by a source follower. We propose a minor change in this architecture, as shown in Figure 1 (top). Two transistors, acting as switches, are added to the 4T architecture in order to connect the floating diffusion nodes of neighboring pixels and to create a reconfigurable array in which every pixel is connected to four of its neighbors. The sensor matrix interconnection pattern can be seen in Figure 1 (bottom). As the switches are closed, pixel averages are computed within every neighborhood in the array. As a first application of the proposed hardware, when all switches are closed an average operation is computed from the entire image, allowing an instant measure of the global luminance of the matrix, which can be used to adjust the dynamic range of the image with algorithms such as the tone mapping. The proposed scheme permits to compute the average of pixels grouped in squares or rectangles with the size defined by the user. This simple operation allows the acceleration of Haar-like features computation, in which the mean value of neighboring rectangles within the image are compared with the goal of detecting a region where there is a high probability of finding a desired

Current-mode analog integrated circuit for focal-plane image compression

The interest in focal-plane processing techniques, by which image processing is carried out at pixel level, has increased since the advent of active pixel sensors in the middle 90s. By sharing processing circuitry by a group of neighboring pixels such techniques enable high-speed imaging operation and massive parallel computation. Focal-plane image compression is particularly interesting, because it allows for further reduction in data rates. The proposed approach also benefits from processing currents rather than voltages, which not only suits current-mode APS imagers, but also enables the circuits to operate at low voltage supply levels and achieve high speed. Moreover, arithmetic computations such as additions and scaling are easily implemented in current mode. Whereas current-mode imaging architectures produce higher fixed pattern noise (FPN) figures than their voltage-mode counterparts, low FPN can be achieved by applying correlated double sampling (CDS) and gain correction techniques. This work presents a 32 × 32 imaging integrated circuit that captures and compresses gray scale images on the focal plane of the image sensors using analog circuits that implement, for every 4 × 4 pixel block, differential pulse-code modulation, linear transform, and vector quantization. Other processing functions implemented in the chip are CDS, analog convolutions and A/D conversion. Theoretical details and circuit designs are described, as well as the test setup of the chip fabricated in a 0.35 μm CMOS process. Experimental results and photographs captured by the chip are shown to validate the technique. The CMOS imager compresses captured images at 0.94 bits/pixel for an overall power consumption below 40 mW (white image), which is equivalent to approximately 36 μW per pixel. Using photographs taken from bar-target pattern inputs, it is shown that details up to 2 cycles/cm are preserved in the decoded images.

Gaussian Pyramid: Comparative Analysis of Hardware Architectures

This paper addresses a comparison of architectures for the hardware implementation of Gaussian image pyramids. Main differences between architectural choices are in the sensor front-end. One side is for architectures consisting of a conventional sensor that delivers digital images and which is followed by digital processors. The other side is for architectures employing a non-conventional sensor with per-pixel embedded preprocessing structures for Gaussian spatial filtering. This later choice belongs to the general category of “artificial retina” sensors which have been for long claimed as potentially advantageous for enhancing throughput and reducing energy consumption of vision systems. These advantages are very important in the internet of things context, where imaging systems are constantly exchanging information. This paper attempts to quantify these potential advantages within a design space in which the degrees of freedom are the number and type of ADCs, single-slope, SAR, cyclic,

Focal-plane image encoder with cascode current mirrors and increased vector quantization bit rate

\Sigma \Delta

High-Level Performance Evaluation of Object Detection Based on Massively Parallel Focal-Plane Acceleration Requiring Minimum Pixel Area Overhead

, and pipeline, and the number of digital processors. Results show that speed and energy advantages of preprocessing sensors are not granted by default and are only realized through proper architectural design. The methodology presented for the comparison between focal-plane and digital approaches is a useful tool for imager design, allowing for the assessment of focal-plane processing advantages.