Bio‐Inspired Artificial Vision and Neuromorphic Image Processing Devices

Abstract

DOI: 10.1002/admt.202100144 processing devices.[1–6] However, conventional image recognition systems using a flat image sensor array with a multilens optical system and the von-Neumann computing architecture for processing the acquired image data have several limitations such as high system-level complexity, bulky module size, large computing load, and low energy efficiency.[7] Therefore, advanced devices in both image acquisition and image data processing are required. As a result, bio-inspired imaging devices[8–10] (i.e., artificial vision) and neuromorphic image processing devices[11–13] (i.e., artificial synapse) have received considerable attention (Figure 1). Bio-inspired imaging devices (e.g., bioinspired camera) have been developed for image acquisition.[14] Conventional imaging devices require bulky and heavy optical systems to obtain high-quality visual information.[15] In contrast, natural eyes have a simple and small optical geometry and high-quality image acquisition capability.[16,17] Therefore, bio-inspired artificial vision has been developed by mimicking the unique structural and functional advantage of natural eyes[2,6] (Figure 1a). For example, the chambered eye, typically found in humans and aquatic animals, exhibits a wide field of view, low optical aberration, and facile accommodation with a simple optical system.[16,17] The compound eye has distinctive optical geometries, and such structures offer various useful visual features.[18,19] Neuromorphic computing devices that can efficiently process massive image data acquired from the imaging device have been developed for image data processing.[20–22] The conventional vonNeumann architecture, in which the central processing unit and memory unit are separated, is not suitable to efficiently process the massive unstructured image data.[23,24] Therefore, a novel computing device inspired by the human brain (i.e., electronic synapse) has been developed[25,26] (Figure 1b). For example, the memristor crossbar array can efficiently perform vector multiplications.[27] Such a neuromorphic device implements artificial neural networks (ANN) in the hardware and enables efficient parallel processing of image data with low energy consumption.[25] In a previous study, a device that integrates the synaptic device and photodetector in one unit has been reported.[26] Despite recent progress in the hardware of the neuromorphic image data processing devices, such devices still require Remarkable technological developments for efficient image recognition (i.e., image acquisition and image data processing) have been reported in the past decade. Such advances in imaging and image processing technologies have driven significant progress in mobile electronics and machine vision applications. In particular, for image acquisition devices, two types of natural eyes (i.e., chambered and compound eyes) have inspired the development of novel multifunctional imaging devices with unique optical geometries. For image data processing devices, novel computing devices based on memristor crossbar arrays, such as electronic synapses, have been developed. More recently, the integration of imaging and image processing devices in a single unit further enhances the system-level efficiency. Herein, such recent advances in the bio-inspired artificial vision and neuromorphic image processing devices, aimed at providing efficient image recognition, are reviewed. First, various imaging devices inspired by the structural and functional features of natural eyes are introduced. Second, artificial synapses and their operation principles are thoroughly discussed. Third, the neuromorphic vision sensor that integrates the imaging and image processing devices is reviewed. Finally, a brief summary and future outlook are presented.