Manuel Suarez

CMOS-3D Smart Imager Architectures for Feature Detection

This paper reports a multi-layered smart image sensor architecture for feature extraction based on detection of interest points. The architecture is conceived for 3-D integrated circuit technologies consisting of two layers (tiers) plus memory. The top tier includes sensing and processing circuitry aimed to perform Gaussian filtering and generate Gaussian pyramids in fully concurrent way. The circuitry in this tier operates in mixed-signal domain. It embeds in-pixel correlated double sampling, a switched-capacitor network for Gaussian pyramid generation, analog memories and a comparator for in-pixel analog-to-digital conversion. This tier can be further split into two for improved resolution; one containing the sensors and another containing a capacitor per sensor plus the mixed-signal processing circuitry. Regarding the bottom tier, it embeds digital circuitry entitled for the calculation of Harris, Hessian, and difference-of-Gaussian detectors. The overall system can hence be configured by the user to detect interest points by using the algorithm out of these three better suited to practical applications. The paper describes the different kind of algorithms featured and the circuitry employed at top and bottom tiers. The Gaussian pyramid is implemented with a switched-capacitor network in less than 50 μs, outperforming more conventional solutions.

Switched-capacitor networks for scale-space generation

In scale-space filtering signals are represented at several scales, each conveying different details of the original signal. Every new scale is the result of a smoothing operator on a former scale. In image processing, scale-space filtering is widely used in feature extractors as the Scale-Invariant Feature Transform (SIFT) algorithm. RC networks are posed as valid scale-space generators in focal-plane processing. Switched-capacitor networks are another alternative, as different topologies and switching rate offer a great flexibility. This work examines the parallel and the bilinear implementations as two different switched-capacitor network topologies for scale-space filtering. The paper assesses the validity of both topologies as scale-space generators in focal-plane processing through object detection with the SIFT algorithm.

A 26.5 nJ/px 2.64 Mpx/s CMOS vision sensor for Gaussian pyramid extraction

This paper introduces a CMOS vision sensor to extract the Gaussian pyramid with an energy cost of 26.5 nJ/px at 2.64 Mpx/s, thus outperforming conventional solutions employing an imager and a separate digital processor. The chip, manufactured in a 0.18 μm CMOS technology, consists of an arrangement of 88 × 60 processing elements (PEs) which captures images of 176 × 120 resolution and performs concurrent parallel processing right at pixel level. The Gaussian pyramid is generated by using a switched-capacitor network. Every PE includes four photodiodes, four MiM capacitors, one 8-bit single-slope ADC and one CDS circuit, occupying 44 × 44 μm2. Suitability of the chip is assessed by using metrics pertaining to visual tracking.

A CMOS-3D reconfigurable architecture with in-pixel processing for feature detectors

This paper introduces a two-tier CMOS-3D architecture for generation of Gaussian pyramids, detection of extrema, and calculation of spatial derivatives in an image. Such tasks are included in modern feature detectors, which in turn can be used for operations like object detection, image registration or tracking. The top tier of the architecture contains the image acquisition circuits in an array of 320 × 240 active photodiode sensors (APS) driving a smaller array of 160 × 120 analog processors for low-level image processing. The top tier comprises in-pixel Correlated Double Sampling (CDS), a switched-capacitor network for Gaussian pyramid generation, analog memories and a comparator for in-pixel Analog to Digital Converter (ADC). The reuse of circuits for different functions permits to have a small area for every pixel. The bottom tier of the architecture contains a frame buffer with a set of registers acting as a frame-buffer with a one-to-one correspondence with the analog processors in the top tier, the digital circuitry necessary for the extrema detection and the calculation of the first and second spatial derivatives in the image, as well as Harris and Hessian point detectors. For the time being, a behavioral model of the first tier including mismatch and feedthrough and charge injection errors is discussed. Also, a VHDL model for the bottom tier is addressed. The two-tier architecture is conceived for its implementation on the 130 nm CMOS-3D technology from Tezzaron. A companion chip will perform the higher-level operations as well as communications. In this technology an area of 300 μm2 per analog processor has been estimated. The architecture proposed for pyramid generation lets a frame rate of 180 frames/s for an ADC conversion time of 120 μs. The architecture has been proved with object detection for a given feature detector.

A 176×120 pixel CMOS vision chip for Gaussian filtering with massivelly Parallel CDS and A/D-conversion

This paper conveys a proof-of-concept chip for Gaussian pyramid generation for image feature detectors. Gaussian filtering and image resizing are performed with a switched-capacitor (SC) network. The chip is conceived as the mapping of a CMOS-3D architecture for feature detectors onto a conventional technology, with some functionality removed, and the corresponding area overhead with respect to that of a CMOS-3D architecture, but preserving masivelly parallel Correlated Double Sampling (CDS) and A/D conversion. The chip has been fabricated on a die of 5×5 mm2 with 0.18 μm CMOS technology, achieving an array of 176×120 sensing elements (pixels). The pixels are arranged in Processing Elements (PEs). Every PE comprises four photodiodes, four SC nodes, one CDS circuit, and local circuitry for one ADC. Every PE occupies an area of 44×44 μm2. The chip senses an image and computes the Gaussian pyramid with an average power consumption lower than 75 nW/pixel at 30 frames/s.

In-pixel ADC for a vision architecture on CMOS-3D technology

This paper addresses the design of an 8-bit single-slope in-pixel ADC for a 3D chip architecture intended for airborne surveillance and reconnaissance applications. The 3D chip architecture comprises a sensor layer with a resolution of 320 × 240 pixels bump-bonded to a three-tier chip on the 150 nm FDSOI CMOS-3D technology from MIT-Lincoln Laboratories. The top tier is a mixed-signal layer with 160 × 120 processing elements. The ADC is distributed between the top two tiers. The top tier contains both global and local circuitry. The ramp generation is implemented with global circuitry through an 8-bit unary current-steering DAC. The end of conversion at every pixel or processing element is triggered by a local comparator. The digital words are stored in a frame-buffer in an intermediate tier. The area of the local circuitry in the ADC is consumed by the comparator, capable of reaching less than 3 mV of resolution in less than 150 ns with less than 220 μm2, and by the memory cells, each one storing 6 8-bit words along with two additional bits in less than 50 μm × 50 μm. Every ADC conversion is performed in less than 120 μs.

Evidence of the lateral collection significance in small CMOS photodiodes

The lateral collection capacity of small CMOS photodiodes, scanned with a point source illumination, is studied. The mathematical solution of the physical equations is compared to experimental measurements in a standard UMC 90nm technology. They show close agreement and reveal that the lateral collection through the sidewalls of the depletion region becomes a significant component of the overall photocurrent. The same conclusion is achieved through device simulations under uniform illumination using ATLAS.

Offset-compensated comparator with full-input range in 150nm FDSOI CMOS-3D technology

This paper addresses an offset-compensated comparator with full-input range in the 150nm FDSOI CMOS-3D technology from MIT- Lincoln Laboratory. The comparator discussed here makes part of a vision system. Its architecture is that of a self-biased inverter with dynamic offset correction. At simulation level, the comparator can reach a resolution of 0.75mV in an area of approximately 220μm2 with a time response of less than 200ns and a static power dissipation of 1.125μW.

Image Feature Extraction Acceleration

Image feature extraction is instrumental for most of the best-performing algorithms in computer vision. However, it is also expensive in terms of computational and memory resources for embedded systems due to the need of dealing with individual pixels at the earliest processing levels. In this regard, conventional system architectures do not take advantage of potential exploitation of parallelism and distributed memory from the very beginning of the processing chain. Raw pixel values provided by the front-end image sensor are squeezed into a high-speed interface with the rest of system components. Only then, after deserializing this massive dataflow, parallelism, if any, is exploited. This chapter introduces a rather different approach from an architectural point of view. We present two Application-Specific Integrated Circuits (ASICs) where the 2-D array of photo-sensitive devices featured by regular imagers is combined with distributed memory supporting concurrent processing. Custom circuitry is added per pixel in order to accelerate image feature extraction right at the focal plane. Specifically, the proposed sensing-processing chips aim at the acceleration of two flagships algorithms within the computer vision community: the Viola-Jones face detection algorithm and the Scale Invariant Feature Transform (SIFT). Experimental results prove the feasibility and benefits of this architectural solution.

Form factor improvement of smart-pixels for vision sensors through 3-D vertically-integrated technologies

While conventional CMOS active pixel sensors embed only the circuitry required for photo-detection, pixel addressing and voltage buffering, smart pixels incorporate also circuitry for data processing, data storage and control of data interchange. This additional circuitry enables data processing be realized concurrently with the acquisition of images which is instrumental to reduce the number of data needed to carry to information contained into images. This way, more efficient vision systems can be built at the cost of larger pixel pitch. Vertically-integrated 3D technologies enable to keep the advnatges of smart pixels while improving the form factor of smart pixels.