Pramod Swami

High Performance Front Camera ADAS Applications on TI's TDA3X Platform

Advanced driver assistance systems (ADAS) are designed to increase drivers situational awareness and road safety by providing essential information, warning and automatic intervention to reduce the possibility/severity of an accident. Of the various types of ADAS modalities available, camera based ADAS are being widely adopted for their usefulness in varied applications, overall reliability and adaptability to new requirements. But camera based ADAS also represents a complex, high-performance, and low-power compute problem, requiring specialized solutions. This paper introduces a high performance front camera ADAS based on a small area, low power System-on-Chip (SoC) solution from Texas Instruments called Texas Instruments Driver Assist 3x (TDA3x). The paper illustrates compute capabilities of the device in implementation of a typical front camera ADAS. The paper also introduces key programming concepts related to heterogeneous programmable compute cores in the SoC and the software framework to use those cores in order to develop the front camera solutions. These aspects will be of interest not only to the ADAS developers but for computer vision and compute intensive embedded system development.

Ultra-low latency video codec for video conferencing

Video codec (e.g. HEVC, H.264, H.263, H.261) are used for real time video conferencing over internet. The amount of latency from end to end (or round trip) has significant impact on perceived quality of video call. This paper explains overall latency for entire signal chain with focus especially on video codec. The paper explains typical configuration to optimize overall latency of video processing down to range of 1 video frame processing time. The paper proposes new sub-frame based data flow to cut down overall latency to significantly to fraction of video frame. The paper proposes new design for video codec engine to enable sub-frame based data flow consisting of novel way of exchanging data between entropy engine and application, pre-fetching of video data without stalling video performance and sending of partial video output to network. The overall design enables reduction of processing latency of video engine from multiple frames to few lines of video. The overall solution on TIs Davinci series (DM816x) device achieves latency up-to 2 msec compared to prior art measurement of 33 msec resulting in better user experience due to large improvements in perceived visual quality.

ORB in 5 ms: An Efficient SIMD Friendly Implementation

One of the key challenges today in computer vision applications is to be able to reliably detect features in real-time. The most prominent feature extraction methods are Speeded up Robust Features(SURF), Scale Invariant Feature Transform(SIFT) and Oriented FAST and Rotated BRIEF(ORB), which have proved to yield reliable features for applications such as object recognition and tracking. In this paper, we propose an efficient single instruction multiple data(SIMD) friendly implementation of ORB. This solution shows that ORB feature extraction can be effectively implemented in about 5.5 ms on a Vector SIMD engine such as Embedded Vision Engine(EVE) of Texas Instruments(TI). We also show that our implementation is reliable with the help of repeatability test.

Sparse, Quantized, Full Frame CNN for Low Power Embedded Devices

This paper presents methods to reduce the complexity of convolutional neural networks (CNN). These include: (1) A method to quickly and easily sparsify a given network. (2) Fine tune the sparse network to obtain the lost accuracy back (3) Quantize the network to be able to implement it using 8-bit fixed point multiplications efficiently. (4) We then show how an inference engine can be designed to take advantage of the sparsity. These techniques were applied to full frame semantic segmentation and the degradation due to the sparsity and quantization is found to be negligible. We show by analysis that the complexity reduction achieved is significant. Results of implementation on Texas Instruments TDA2x SoC [17] are presented. We have modified Caffe CNN framework to do the sparse, quantized training described in this paper. The source code for the training is made available at https://github.com/tidsp/caffe-jacinto

A Diverse Low Cost High Performance Platform for Advanced Driver Assistance System (ADAS) Applications

Advanced driver assistance systems (ADAS) are becoming more and more popular. Lot of the ADAS applications such as Lane departure warning (LDW), Forward Collision Warning (FCW), Automatic Cruise Control (ACC), Auto Emergency Braking (AEB), Surround View (SV) that were present only in high-end cars in the past have trickled down to the low and mid end vehicles. Lot of these applications are also mandated by safety authorities such as EUNCAP and NHTSA. In order to make these applications affordable in the low and mid end vehicles, it is important to have a cost effective, yet high performance and low power solution. Texas Instruments (TIs) TDA3x is an ideal platform which addresses these needs. This paper illustrates mapping of different algorithms such as SV, LDW, Object detection (OD), Structure From Motion (SFM) and Camera-Monitor Systems (CMS) to the TDA3x device, thereby demonstrating its compute capabilities. We also share the performance for these embedded vision applications, showing that TDA3x is an excellent high performance device for ADAS applications.

Efficient object detection and classification on low power embedded systems

Identifying real world 3D objects such as pedestrians, vehicles and traffic signs using 2D images is a challenging task. There are multiple approaches to tackle this problem with varying degree of detection accuracy and implementation complexity. Some approaches use “hand coded” object features such as Histogram of Oriented Gradients (HOG), Haar, Scale Invariant Feature Transform (SIFT) along with a linear classifier such as Support Vector Machine (SVM), Adaptive Boosting (AdaBoost) to detect objects. Recent developments have shown that a deep multi-layered Convolution Neural Network (CNN) classifier can learn the object features on its own and also classify at an accuracy surpassing human vision. In this paper we combine both the approaches; “object detection” is done using HOG features and AdaBoost cascade classifier and “object classification” is done using CNN to classify the type of objects being detected. The proposed method is implemented on TIs low power TDA3x SoC.

A robust and real-time image based lane departure warning system

Lane departure warning (LDW) systems have gained a lot of interest over the last few years. EuroNCAP regulations mandate European car makers to have LDW system to get star rating. In this paper, we propose a simple image-based implementation of LDW system optimized for the Texas Instruments (TI) C66x Digital Signal Processor (DSP). We show that our solution is able to overcome certain challenges faced in prior methods and consumes only 2.3 mega cycles/frame of the DSP, leaving the rest of the DSP for other applications. We also share some results of our implementation.

Multichannel video software solution for an asynchronous multiprocessor system

Multichannel video usecases put considerable resource demands on the processing system. Software designed for single channel when extended to multichannel often does not exploit the capabilities of the system fully and limits the number of channels that can be processed simultaneously. In this paper, we discuss the performance constraints observed in an asynchronous multiprocessor system when extending a single channel codec software solution to a multichannel usecase. This is followed by a discussion on building a multichannel codec solution using the existing single channel framework by exploiting bunch submission and parallelizing the operations of different processing cores to maximize the system resource utilization. Upto 30% channel density gains are observed when migrating from the single channel to multichannel solution.

Real time Structure from Motion for Driver Assistance System

Understanding of 3D surrounding is an important problem in Advanced Driver Assistance Systems (ADAS). Structure from Motion (SfM) is well known computer vision technique for estimating 3D structure from 2D image sequences. Inherent complexities of the SfM pose different algorithmic and implementation challenges to have an efficient enablement on embedded processor for real time processing. This paper focuses on highlighting such challenges and innovative solutions for them. The paper proposes an efficient SfM solution that has been implemented on Texas Instruments TDA3x series of System on Chip (SoC). The TDA3x SoC contains one vector processor (known as EVE) and two C66x DSPs as co-processors which are useful for computationally intensive vision processing. The proposed SfM solution which performs Sparse Optical Flow, Fundamental matrix estimation, Triangulation, 3D points pruning consumes 42% of EVE and 10% of one DSP for 25 fps processing of one mega pixel image resolution.

High performance automotive radar signal processing on TI's TDA3X platform

Automotive is an important application of radar. In collision avoidance applications, radar and camera are two main sensors with radar having a healthy share [16]. Usage of radar in the automotive space is expanding beyond range and velocity detection of obstacles to more sophisticated usage of object motion direction estimation [17], precise angular position estimation of obstacles in urban environments [6][7] and ground vehicle localization[15]. As a result, radar processing solutions require complex signal processing with higher programmability. In this paper, we explain the mapping of automotive MIMO radar processing chain on TIs TDA3x platform and highlight the need for a heterogeneous processor architecture with adequate programmability.