Cong Shi

7.3 A 1000fps vision chip based on a dynamically reconfigurable hybrid architecture comprising a PE array and self-organizing map neural network

A vision chip is a high-speed and compact vision system that integrates an image sensor and parallel image processors on a single silicon die. Nowadays, high-speed vision chips with powerful recognition capabilities are greatly demanded in applications such as: industrial automation, security, entertainment, robotic vision, and human-machine interaction. Some 100-to-1,000fps vision chips have been reported [1-4]. These chips integrate pixel-parallel and row-parallel SIMD array processors to speed up low- and mid-level image processing [1,2]. Recently, microprocessors (MPU) have been embedded to carry out high-level image processing [3,4]. Although excellent in low- and mid-level processing, these systems are poor in high-level feature vector (FV) recognition tasks due to the von Neumann bottleneck of the MPU. As a consequence, these chips can no longer achieve 1,000fps system-level performance, from image acquisition to high-level feature-recognition processing.

A 10-bit column-parallel cyclic ADC for high-speed CMOS image sensors

This paper presents a high-speed column-parallel cyclic analog-to-digital converter (ADC) for a CMOS image sensor. A correlated double sampling (CDS) circuit is integrated in the ADC, which avoids a stand-alone CDS circuit block. An offset cancellation technique is also introduced, which reduces the column fixed-pattern noise (FPN) effectively. One single channel ADC with an area less than 0.02 mm2 was implemented in a 0.13 μm CMOS image sensor process. The resolution of the proposed ADC is 10-bit, and the conversion rate is 1.6 MS/s. The measured differential nonlinearity and integral nonlinearity are 0.89 LSB and 6.2 LSB together with CDS, respectively. The power consumption from 3.3 V supply is only 0.66 mW. An array of 48 10-bit column-parallel cyclic ADCs was integrated into an array of CMOS image sensor pixels. The measured results indicated that the ADC circuit is suitable for high-speed CMOS image sensors.

A low power global shutter pixel with extended FD voltage swing range for large format high speed CMOS image sensor

A low power 8T global shutter pixel with extended FD voltage swing range is proposed for large format high speed CMOS image sensor. The pixel has a negative threshold reset transistor, two in-pixel source followers, and a sample-and-hold circuit. The in-pixel first source follower is employed for reducing the pixel average current and maximum transient current. The negative threshold reset transistor is applied to extend the voltage swing of FD. Using pixel level sample-and-hold circuit, the kTC noise on FD node can be effectively nullified by correlated double sampling operation. A high speed 1000 fps 256 × 256 CMOS image sensor is implemented in 0.18 μm CMOS process. Two 10-bit cyclic ADC arrays are integrated in this prototype sensor chip. The active area of the chip is 10 mm × 7 mm with a pixel size of 14 μm × 14 μm. The developed sensor achieves an average current of 23 nA per pixel, a maximum transit current per pixel as low as 1113 nA, and a large FD voltage swing of 1.78 V. The sensor temporal noise level is 103 e- and full well capacity has 27000 e-which results in 48.3 dB signal dynamic range

A high speed multi-level-parallel array processor for vision chips

This paper proposes a high speed multi-level-parallel array processor for programmable vision chips. This processor includes 2-D pixel-parallel processing element (PE) array and 1-D row-parallel row processor (RP) array. The two arrays both operate in a single-instruction multiple-data (SIMD) fashion and share a common instruction decoder. The sizes of the arrays are scalable according to dedicated applications. In PE array, each PE can communicate not only with its nearest neighbor PEs, but also with the next near neighbor PEs in diagonal directions. This connection can help to speed up local operations in low-level image processing. On the other hand, global operations in mid-level processing are accelerated by the skipping chain and binary boosters in RP array. The array processor was implemented on an FPGA device, and was successfully tested for various algorithms, including real-time face detection based on PPED algorithm. The results show that the image processing speed of proposed processor is much higher than that of the state-of-the-arts digital vision chips.

A novel architecture of local memory for programmable SIMD vision chip

This paper presents a novel architecture of the local memory for the programmable SIMD vision chip. The memory architecture consists of 8 × 8 local memory cells, among which each 8 static latches in the master stage share one dynamic latch in the slave stage. The local memory performs single bit read and write in each clock cycle, and the compact area of 14.33 μm2/bit increases the integration level of the processor. A prototype chip with 64 × 64 processing units has been manufactured in 0.18 μm CMOS technology. Five types of local memory architecture have been designed, and an 8-bit input data buffer based on dedicated latch structures has been designed as the input data buffer for each processing unit. Test results show that the presented structure is suitable for real-time computer vision applications such as the edge detection at the speed of 1000 fps.

A high-speed vision processor based on pixel-parallel PE array and its applications

This paper proposes a novel high-speed vision processor based on pixel-parallel PE array. The processor consists of a pixel-parallel PE array, an embedded RISC core, an AHB bus, some SRAM blocks and other logical controllers. PE array performs the low- and mid-level vision processing, and RISC core carries out the high-level vision processing in succession. The vision processor can process vision information at a speed higher than 1000fps. Many application algorithms can be implemented by software programming on the vision processor. The processor prototype is implemented on a FPGA board with a 64×64 PE array and the clock frequency is 100MHz. It can realize moving detection in 128×128 image data at a rate of 4500fps, and 104fps in complicated face detection task for 160×120 video frame sequence. The results demonstrated that the vision processor outperforms the general serial CPUs for more than 60 times.

High speed vision processor with reconfigurable processing element array based on full-custom distributed memory

In this paper, a hybrid vision processor based on a compact full-custom distributed memory for near-sensor high-speed image processing is proposed. The proposed processor consists of a reconfigurable processing element (PE) array, a row processor (RP) array, and a dual-core microprocessor. The PE array includes two-dimensional processing elements with a compact full-custom distributed memory. It supports real-time reconfiguration between the PE array and the self-organized map (SOM) neural network. The vision processor is fabricated using a 0.18 µm CMOS technology. The circuit area of the distributed memory is reduced markedly into 1/3 of that of the conventional memory so that the circuit area of the vision processor is reduced by 44.2%. Experimental results demonstrate that the proposed design achieves correct functions.

A massively parallel keypoint detection and description (MP-KDD) algorithm for high-speed vision chip

This paper proposes a massively parallel keypoint detection and description (MP-KDD) algorithm for the vision chip with parallel array processors. The MP-KDD algorithm largely reduces the computational overhead by removing all floating-point and multiplication operations while preserving the currently popular SIFT and SURF algorithm essence. The MP-KDD algorithm can be directly and effectively mapped onto the pixel-parallel and row-parallel array processors of the vision chip. The vision chip architecture is also enhanced to realize direct memory access (DMA) and random access to array processors so that the MP-KDD algorithm can be executed more effectively. An FPGA-based vision chip prototype is implemented to test and evaluate our MP-KDD algorithm. Its image processing speed reaches 600–760 fps with high accuracy for complex vision applications, such as scene recognition.摘要本文提出了一种面向视觉芯片并行图像处理阵列的高效图像特征点提取和描述算法。 该算法基于SIFT特征点检测及SURF特征点描述, 但简化避免了浮点运算以及乘除法操作, 极大地节约了硬件开销和处理时间。 该算法可以直接、 高效地映射到视觉芯片的像素级并行和行并行阵列处理器。 本文实现了基于FPGA的视觉原型, 在其上成功测试了所提出的算法, 达到了600–700帧/秒的较高速度。

Smart image sensing system

Vision chips have achieved excellent performance in machine vision applications, but it is insufficient to most industrial applications due to low resolution image sensors and poor image quality. To solve the problems, a smart image sensing system which integrates image sensor and vision chip architecture image processor is proposed. The CMOS image sensor consists of an 800×600 pixel array and two column-parallel ADCs. It is capable of capturing images at 1000 fps. The image processor contains a 64 × 64 processing elements array, 64 row processors, and dual-core RISC. It can exploit data-level parallelism, therefore massively accelerate both low- and middle-level image processing. The proposed image sensing system is successfully applied to various applications like edge detection, motion detection, target tracking at a processing rate of 1000 fps.

A low-power column-parallel ADC for high-speed CMOS image sensor

This paper presents a 10-bit low-power column-parallel cyclic analog-to-digital converter (ADC) used for high-speed CMOS image sensor (CIS). An opamp sharing technique is used to save power and area. Correlated double sampling (CDS) circuit and programmable gain amplifier (PGA) are integrated in the ADC, which avoids stand-alone circuit blocks. An offset cancellation technique is also introduced, which reduces the column fixed-pattern noise (FPN) effectively. One single channel ADC with an area less than 0.03mm2 was implemented in a 0.18μm 1P4M CMOS image sensor process. The resolution of the proposed ADC is 10-bit, and the conversion rate is 2MS/s. The measured differential nonlinearity (DNL) and integral nonlinearity (INL) are 0.62 LSB and 2.1 LSB together with CDS, respectively. The power consumption from 1.8V supply is only 0.36mW.