Emanuele Plebani

Reduced memory region based deep Convolutional Neural Network detection

Accurate pedestrian detection has a primary role in automotive safety: for example, by issuing warnings to the driver or acting actively on cars brakes, it helps decreasing the probability of injuries and human fatalities. In order to achieve very high accuracy, recent pedestrian detectors have been based on Convolutional Neural Networks (CNN). Unfortunately, such approaches require vast amounts of computational power and memory, preventing efficient implementations on embedded systems. This work proposes a CNN-based detector, adapting a general-purpose convolutional network to the task at hand. By thoroughly analyzing and optimizing each step of the detection pipeline, we develop an architecture that outperforms methods based on traditional image features and achieves an accuracy close to the state-of-the-art while having low computational complexity. Furthermore, the model is compressed in order to fit the tight constrains of low power devices with a limited amount of embedded memory available. This paper makes two main contributions: (1) it proves that a region based deep neural network can be finely tuned to achieve adequate accuracy for pedestrian detection (2) it achieves a very low memory usage without reducing detection accuracy on the Caltech Pedestrian dataset.

Visual Search of multiple objects from a single query

Hundreds of millions of images are uploaded to the cloud every day. Innovative applications able to analyze and extract efficiently information from such a big database are needed nowadays more than ever. Visual Search is an application able to retrieve information of a query image comparing it against a large image database. In this paper a Visual Search pipeline implementation is presented able to retrieve multiple objects depicted in a single query image. Quantitative and qualitative precision results are shown on both real and synthetic datasets.

Mixing retrieval and tracking using compact visual descriptors

Visual search has seen many improvements over the years, but its application on video content is still an open research problem and it is often limited to still images. Based on the tools devised by the standardization group MPEG CDVS, we developed a processing flow that processes at nearly real-time a video acquired with a low cost imager and performs content search and retrieval of the top match from a local database. To allow efficient interest point detection, we used a GPU accelerated SIFT library. To process the video frames efficiently, we developed a new dataflow processing which allows switching between object searching, retrieving and tracking in order to keep at minimum the number of queries sent to the database. A search into a local database is performed only when no object has been recognized, and once a good match has been found, the algorithm switches to tracking mode.

Embedded Real-Time Visual Search with Visual Distance Estimation

Visual Search algorithms are a class of methods that retrieve images by their content. In particular, given a database of reference images and a query image the goal is to find an image among the database that depicts the same object as in the query, if any. Moreover, in many different real case applications more than one object of interest could be viewed in the query image. Furthermore, in this kind of situations, often, it is not sufficient to identify the object depicted on a query image but its precise localization inside the scene viewed by the camera is also requested. In this paper we propose to couple a Visual Search system, which can retrieve multiple objects from the same query image, with an additional Distance Estimation module that exploits the localization information already computed inside the Visual Search stage to estimate localization of the object in three dimensions. In this work we implement the complete image retrieval and spatial localization pipeline (including relative distance estimation) on two different embedded devices, exploiting also their GPU in order to get near real time performances on low-power devices. Lastly, the accuracy of the proposed distance estimation is evaluated on a dataset of annotated query-reference pairs ad-hoc created.

Complexity and Accuracy of Hand-Crafted Detection Methods Compared to Convolutional Neural Networks

Even though Convolutional Neural Networks have had the best accuracy in the last few years, they have a price in term of computational complexity and memory footprint, due to a large number of multiply-accumulate operations and model parameters. For embedded systems, this complexity severely limits the opportunities to reduce power consumption, which is dominated by memory read and write operations. Anticipating the oncoming integration into intelligent sensor devices, we compare hand-crafted features for the detection of a limited number of objects against some typical convolutional neural network architectures. Experiments on some state-of-the-art datasets, addressing detection tasks, show that for some problems the increased complexity of neural networks is not reflected by a large increase in accuracy. Moreover, our analysis suggests that for embedded devices hand-crafted features are still competitive in terms of accuracy/complexity trade-offs.

Accurate cyber-physical system simulation for distributed visual search applications

A Cyber-Physical System (CPS) is defined as a, usually distributed, system that links the digital (cyber) and physical world. They feature different computational cores and heterogeneous sensors linked through networks of different types allowing a deeper interaction with the physical world, collecting, storing and exchanging information intelligently. In this work, an open source CPS simulator called COSSIM is described and a smart mechanism is proposed in order to turn it into a co-simulator. In addition to this, a CPS based on a Computer Vision application, called Mobile Visual Search (MVS), is described and ported to COSSIM in order to test the correctness of the simulation and to prove the benefit of the proposed acceleration. Quantitative and qualitative precision results in both real and simulated scenarios are also presented.

Accurate omnidirectional multi-camera embedded structure from motion

Trajectory estimation and 3D scene reconstruction from multiple cameras (also referred as Structure from Motion, SfM) will have a central role in the future of automotive industry. Typical appliance fields will be: autonomous navigation/guidance, collisions avoidance against static or moving objects (in particular pedestrians), parking assisted maneuvers and many more. The work exposed in this paper had mainly two different goals: (1) to describe the implementation of a real time embedded SfM modular pipeline featuring a dedicated optimized HW/SW system partitioning. It included also nonlinear optimizations such as local and global bundle adjustment at different stages of the pipeline; (2) to demonstrate quantitatively its performances on a synthetic test space specifically designed for its characterization. In order to make the system reliable and effective, providing the driver or the autonomous vehicle with a prompt response, the data rates and low latency of the 5G communication systems appear to make this choice the most promising communication solution.

The Orlando Project: A 28 nm FD-SOI Low Memory Embedded Neural Network ASIC

The recent success of neural networks in various computer vision tasks open the possibility to add visual intelligence to mobile and wearable devices; however, the stringent power requirements are unsuitable for networks run on embedded CPUs or GPUs. To address such challenges, STMicroelectronics developed the Orlando Project, a new and low power architecture for convolutional neural network acceleration suited for wearable devices. An important contribution to the energy usage is the storage and access to the neural network parameters. In this paper, we show that with adequate model compression schemes based on weight quantization and pruning, a whole AlexNet network can fit in the local memory of an embedded processor, thus avoiding additional system complexity and energy usage, with no or low impact on the accuracy of the network. Moreover, the compression methods work well across different tasks, e.g. image classification and object detection.

Accurate characterization of embedded Structure from Motion

Trajectory estimation and 3d scene reconstruction from single camera, e.g. Structure from Motion, is going to have a central role in the future of automotive industry. Typical appliance fields will be: collisions avoidance with any kind of object (people included), parking assisted maneuvers and many more. Indeed various countries are becoming more and more concerned about road traffic safety and therefore through its “Advanced Program”, EuroNCAP rewards vehicle manufacturers who employ Advanced Safety Technologies that assists the driver. This paper had mainly two different goals: (1) to describe the implementation of a state of art Structure from Motion pipeline able to run in real time with embedded fish-eye camera, which includes nonlinear optimization (i.e. local bundle adjustment); (2) to demonstrate quantitatively its performances on a synthetic test space specifically designed for its characterization in term of accuracy.

Training an object detector using only positive samples

Accurate pedestrian detection has an important role in automotive applications because, by issuing warnings to the driver and acting actively on the car brakes, it can save human lives and decrease the probability of injuries. In order to achieve adequate accuracy, detectors require training sets containing a very large number of negative samples, which can be challenging for the training algorithms of models like support vector machines (SVM). A common approach to deal with such large datasets is Hard Negative Mining (HNM), which avoids working on the full set by growing an active pool of mined samples. A more recent method is the Block-Circulant Decomposition, which achieves the accuracy of HNM at a lower computational cost by reformulating the problem in the Fourier domain. The method however results in additional memory, required during training by the FFT transform, which could be reduced significantly by using only the positive examples. To address the problem, this paper proposes two main contributions: (1) it shows that the circulant decomposition method works with the same performances when only the positive samples are used in the training phase (2) it compares the performance of a detection pipeline based on HOG features trained with either both all negative and positive samples or with only positive samples on the INRIA pedestrian dataset.