Ricardo Carmona-Galán

ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs

Today, with 0.18-/spl mu/m technologies mature and stable enough for mixed-signal design with a large variety of CMOS compatible optical sensors available and with 0.09-/spl mu/m technologies knocking at the door of designers, we can face the design of integrated systems, instead of just integrated circuits. In fact, significant progress has been made in the last few years toward the realization of vision systems on chips (VSoCs). Such VSoCs are eventually targeted to integrate within a semiconductor substrate the functions of optical sensing, image processing in space and time, high-level processing, and the control of actuators. The consecutive generations of ACE chips define a roadmap toward flexible VSoCs. These chips consist of arrays of mixed-signal processing elements (PEs) which operate in accordance with single instruction multiple data (SIMD) computing architectures and exhibit the functional features of CNN Universal Machines. They have been conceived to cover the early stages of the visual processing path in a fully-parallel manner, and hence more efficiently than DSP-based systems. Across the different generations, different improvements and modifications have been made looking to converge with the newest discoveries of neurobiologists regarding the behavior of natural retinas. This paper presents considerations pertaining to the design of a member of the third generation of ACE chips, namely to the so-called ACE16k chip. This chip, designed in a 0.35-/spl mu/m standard CMOS technology, contains about 3.75 million transistors and exhibits peak computing figures of 330 GOPS, 3.6 GOPS/mm/sup 2/ and 82.5 GOPS/W. Each PE in the array contains a reconfigurable computing kernel capable of calculating linear convolutions on 3/spl times/3 neighborhoods in less than 1.5 /spl mu/s, imagewise Boolean combinations in less than 200 ns, imagewise arithmetic operations in about 5 /spl mu/s, and CNN-like temporal evolutions with a time constant of about 0.5 /spl mu/s. Unfortunately, the many ideas underlying the design of this chip cannot be covered in a single paper; hence, this paper is focused on, first, placing the ACE16k in the ACE chip roadmap and, then, discussing the most significant modifications of ACE16K versus its predecessors in the family.

FLIP-Q: A QCIF Resolution Focal-Plane Array for Low-Power Image Processing

This paper reports a 176×144-pixel smart image sensor designed and fabricated in a 0.35 CMOS-OPTO process. The chip implements a massively parallel focal-plane processing array which can output different simplified representations of the scene at very low power. The array is composed of pixel-level processing elements which carry out analog image processing concurrently with photosensing. These processing elements can be grouped into fully-programmable rectangular-shape areas by loading the appropriate interconnection patterns into the registers at the edge of the array. The targeted processing can be thus performed block-wise. Readout is done pixel-by-pixel in a random access fashion. On-chip 8b ADC is provided. The image processing primitives implemented by the chip, experimentally tested and fully functional, are scale space and Gaussian pyramid generation, fully-programmable multiresolution scene representation-including foveation-and block-wise energy-based scene representation. The power consumption associated to the capture, processing and A/D conversion of an image flow at 30 fps, with full-frame processing but reduced frame size output, ranges from 2.7 mW to 5.6 mW, depending on the operation to be performed.

Early forest fire detection by vision-enabled wireless sensor networks

Wireless sensor networks constitute a powerful technology particularly suitable for environmental monitoring. With regard to wildfires, they enable low-cost fine-grained surveillance of hazardous locations like wildland–urban interfaces. This paper presents work developed during the last 4 years targeting a vision-enabled wireless sensor network node for the reliable, early on-site detection of forest fires. The tasks carried out ranged from devising a robust vision algorithm for smoke detection to the design and physical implementation of a power-efficient smart imager tailored to the characteristics of such an algorithm. By integrating this smart imager with a commercial wireless platform, we endowed the resulting system with vision capabilities and radio communication. Numerous tests were arranged in different natural scenarios in order to progressively tune all the parameters involved in the autonomous operation of this prototype node. The last test carried out, involving the prescribed burning of a 95 × 20-m shrub plot, confirmed the high degree of reliability of our approach in terms of both successful early detection and a very low false-alarm rate.

CMOS-3D Smart Imager Architectures for Feature Detection

This paper reports a multi-layered smart image sensor architecture for feature extraction based on detection of interest points. The architecture is conceived for 3-D integrated circuit technologies consisting of two layers (tiers) plus memory. The top tier includes sensing and processing circuitry aimed to perform Gaussian filtering and generate Gaussian pyramids in fully concurrent way. The circuitry in this tier operates in mixed-signal domain. It embeds in-pixel correlated double sampling, a switched-capacitor network for Gaussian pyramid generation, analog memories and a comparator for in-pixel analog-to-digital conversion. This tier can be further split into two for improved resolution; one containing the sensors and another containing a capacitor per sensor plus the mixed-signal processing circuitry. Regarding the bottom tier, it embeds digital circuitry entitled for the calculation of Harris, Hessian, and difference-of-Gaussian detectors. The overall system can hence be configured by the user to detect interest points by using the algorithm out of these three better suited to practical applications. The paper describes the different kind of algorithms featured and the circuitry employed at top and bottom tiers. The Gaussian pyramid is implemented with a switched-capacitor network in less than 50 μs, outperforming more conventional solutions.

An 0.5-/spl mu/m CMOS analog random access memory chip for TeraOPS speed multimedia video processing

Data compressing, data coding, and communications in object-oriented multimedia applications like telepresence, computer-aided medical diagnosis, or telesurgery require an enormous computing power-in the order of trillions of operations per second (TeraOPS). Compared with conventional digital technology, cellular neural/nonlinear network (CNN)-based computing is capable of realizing these TeraOPS-range image processing tasks in a cost-effective implementation. To exploit the computing power of the CNN Universal Machine (CNN-UM), the CNN chipset architecture has been developed-a mixed-signal hardware platform for CNN-based image processing. One of the nonstandard components of the chipset is the cache memory of the analog array processor, the analog random access memory (ARAM). This paper reports on an ARAM chip that has been designed and fabricated in a 0.5-/spl mu/m CMOS technology. This chip consists of a fully addressable array of 32/spl times/256 analog memory registers and has a packing density of 637 analog-memory-cells/mm/sup 2/. Random and nondestructive access of the memory contents is available. Bottom-plate sampling techniques have been employed to eliminate harmonic distortion introduced by signal-dependent feedthrough. Signal coupling and interaction have been minimized by proper layout measures, including the use of protection rings and separate power supplies for the analog and the digital circuitry. This prototype features an equivalent resolution of up to 7 bits-measured by comparing the reconstructed waveform with the original input signal. Measured access times for writing/reading to/from the memory registers are of 200 ns. I/O rates via the l6-line-wide I/O bus exceed 10 Msamples/s. Storage time at room temperature is in the 80 to 100 ms range, without accuracy loss.

Focal-Plane Sensing-Processing: A Power-Efficient Approach for the Implementation of Privacy-Aware Networked Visual Sensors

The capture, processing and distribution of visual information is one of the major challenges for the paradigm of the Internet of Things. Privacy emerges as a fundamental barrier to overcome. The idea of networked image sensors pervasively collecting data generates social rejection in the face of sensitive information being tampered by hackers or misused by legitimate users. Power consumption also constitutes a crucial aspect. Images contain a massive amount of data to be processed under strict timing requirements, demanding high-performance vision systems. In this paper, we describe a hardware-based strategy to concurrently address these two key issues. By conveying processing capabilities to the focal plane in addition to sensing, we can implement privacy protection measures just at the point where sensitive data are generated. Furthermore, such measures can be tailored for efficiently reducing the computational load of subsequent processing stages. As a proof of concept, a full-custom QVGA vision sensor chip is presented. It incorporates a mixed-signal focal-plane sensing-processing array providing programmable pixelation of multiple image regions in parallel. In addition to this functionality, the sensor exploits reconfigurability to implement other processing primitives, namely block-wise dynamic range adaptation, integral image computation and multi-resolution filtering. The proposed circuitry is also suitable to build a granular space, becoming the raw material for subsequent feature extraction and recognition of categorized objects.

A prototype node for wireless vision sensor network applications development

This paper presents a prototype vision-enabled sensor node based on a commercial vision system of reduced size and power consumption. The wireless infrastructure for the deployment of a distributed smart camera network based on these nodes is provided by commercial motes. The smart camera, based on a low-power bio-inspired processing scheme, enables in-node image processing and vision tools. This permits to elaborate a lighter representation of the scene, keeping the relevant information in terms of detected elements, features and events, alleviating the data transmission through the network. Therefore by passing only the relevant information to the neighboring sensor nodes, distributed and collaborative vision is possible with the limited data rates available in commercial wireless sensor networks. Communication between the different components of the system is supported by the available UARTs and GPIOs. Several examples of in-node image processing and feature detection has been tested in the prototype, and information at different abstraction levels has been broadcasted to the network.

All-MOS implementation of RC networks for time-controlled Gaussian spatial filtering

This paper addresses the design and VLSI implementation of MOS-based RC networks capable of performing time-controlled Gaussian filtering. In these networks, all the resistors are substituted one by one by a single MOS transistor biased in the ohmic region. The design of this elementary transistor is carefully realized according to the value of the ideal resistor to be emulated. For a prescribed signal range, the MOSFET in triode region delivers an interval of instantaneous resistance values. We demonstrate that, for the elementary 2-node network, establishing the design equation at a particular point within this interval guarantees minimum error. This equation is then corroborated for networks of arbitrary size by analyzing them from a stochastic point of view. Following the design methodology proposed, the error committed by an MOS-based grid when compared with its equivalent ideal RC network is, despite the intrinsic nonlinearities of the transistors, below 1% even under mismatch conditions of 10%. In terms of image processing, this error hardly affects the outcome, which is perceptually equivalent to that of the ideal network. These results, extracted from simulation, are verified in a prototype vision chip with QCIF resolution manufactured in the AMS 0.35µm CMOS-OPTO process. This prototype incorporates a focal-plane MOS-based RC network that performs fully programmable Gaussian filtering. Copyright

Wi-FLIP: A wireless smart camera based on a focal-plane low-power image processor

This paper presents Wi-FLIP, a vision-enabled WSN node resulting from the integration of FLIP-Q, a prototype vision chip, and Imotel, a commercial WSN platform. In Wi-FLIP, image processing is not only constrained to the digital domain like in conventional architectures. Instead, its image sensor — the FLIP-Q prototype — incorporates pixel-level processing elements (PEs) implemented by analog circuitry. These PEs are interconnected, rendering a massively parallel SIMD-based focal-plane array. Low-level image processing tasks fit very well into this processing scheme. They feature a heavy computational load composed of pixel-wise repetitive operations which can be realized in parallel with moderate accuracy. In such circumstances, analog circuitry, not very precise but faster and more area- and power-efficient than its digital counterpart, has been extensively reported to achieve better performance. The Wi-FLIPs image sensor does not therefore output raw but pre-processed images that make the subsequent digital processing much lighter. The energy cost of such pre-processing is really low — 5.6mW for the worst-case scenario. As a result, for the configuration where the Imote2s processor works at minimum clock frequency, the maximum power consumed by our prototype represents only the 5.2% of the whole system power consumption. This percentage gets even lower as the clock frequency increases. We report experimental results for different algorithms, image resolutions and clock frequencies. The main drawback of this first version of Wi-FLIP is the low frame rate reachable due to the non-standard GPIO-based FLIPQ-to-Imote2 interface.

Switched-capacitor networks for scale-space generation

In scale-space filtering signals are represented at several scales, each conveying different details of the original signal. Every new scale is the result of a smoothing operator on a former scale. In image processing, scale-space filtering is widely used in feature extractors as the Scale-Invariant Feature Transform (SIFT) algorithm. RC networks are posed as valid scale-space generators in focal-plane processing. Switched-capacitor networks are another alternative, as different topologies and switching rate offer a great flexibility. This work examines the parallel and the bilinear implementations as two different switched-capacitor network topologies for scale-space filtering. The paper assesses the validity of both topologies as scale-space generators in focal-plane processing through object detection with the SIFT algorithm.