Zoran Nikolic

Design and implementation of numerical linear algebra algorithms on fixed point DSPs

Numerical linear algebra algorithms use the inherent elegance of matrix formulations and are usually implemented using C/C++ floating point representation. The system implementation is faced with practical constraints because these algorithms usually need to run in real time on fixed point digital signal processors (DSPs) to reduce total hardware costs. Converting the simulation model to fixed point arithmetic and then porting it to a target DSP device is a difficult and time-consuming process. In this paper, we analyze the conversion process. We transformed selected linear algebra algorithms from floating point to fixed point arithmetic, and compared real-time requirements and performance between the fixed point DSP and floating point DSP algorithm implementations. We also introduce an advanced code optimization and an implementation by DSP-specific, fixed point C code generation. By using the techniques described in the paper, speed can be increased by a factor of up to 10 compared to floating point emulation on fixed point hardware.

Algorithmic and software techniques for embedded vision on programmable processors

In the last few years, programmable architectures centered around high-end DSP processors have emerged as the platform of choice for high-volume embedded vision applications, such as automotive safety and video surveillance. Their programmability inherently addresses the problems presented by the sheer diversity of vision algorithms. This paper provides an overview of high-impact algorithmic and software techniques for embedded vision applications implemented on programmable architectures and discusses several system-level issues. We provide a general discussion and practical examples for the following categories of algorithmic techniques: fast algorithms, reduced dimensionality and mathematical shortcuts. Additionally, we discuss the importance of software techniques such as the use of fixed-point arithmetic, reduced data transfers and cache-friendly programming. In our experience, each of these techniques is a key enabler for real-time embedded vision systems.

High Performance Front Camera ADAS Applications on TI's TDA3X Platform

Advanced driver assistance systems (ADAS) are designed to increase drivers situational awareness and road safety by providing essential information, warning and automatic intervention to reduce the possibility/severity of an accident. Of the various types of ADAS modalities available, camera based ADAS are being widely adopted for their usefulness in varied applications, overall reliability and adaptability to new requirements. But camera based ADAS also represents a complex, high-performance, and low-power compute problem, requiring specialized solutions. This paper introduces a high performance front camera ADAS based on a small area, low power System-on-Chip (SoC) solution from Texas Instruments called Texas Instruments Driver Assist 3x (TDA3x). The paper illustrates compute capabilities of the device in implementation of a typical front camera ADAS. The paper also introduces key programming concepts related to heterogeneous programmable compute cores in the SoC and the software framework to use those cores in order to develop the front camera solutions. These aspects will be of interest not only to the ADAS developers but for computer vision and compute intensive embedded system development.

Implementation considerations for single-camera steering assistance systems on a fixed point DSP

The design flow of camera-based steering assistance algorithms usually begins with their implementation in floating-point on a PC or workstation. This abstraction from all implementation effects allows an exploration of the algorithm space. Memory, throughput and word-length requirements may not be important issues for offline implementation of the algorithms, but they can become critical issues for real-time implementations on embedded processors. The implementation of driver assistance systems is faced with practical constraints because these algorithms usually need to run in real-time on fixed point digital signal processors (DSPs) to reduce total hardware cost. In this paper we first evaluate numerical requirements for implementation of camera-based lateral position detection algorithms, such as lane keep assistant and lane departure warning. We then present methods that address the challenges and requirements of fixed-point design process. The flow proposed is targeted at converting C/C++ code with floating-point operations into C code with integer operations that can then be fed through the native C compiler for a fixedpoint DSP. We demonstrate the flow on tracking example (extended Kalman filter) using synthetically generated data, and we analyze trade-offs for algorithm implementation in fixed-point arithmetic.

Embedded Vision in Advanced Driver Assistance Systems

Throughout history, advances in transportation systems have had large economic and cultural impact. Mobility has changed the way people live and automobiles continue to evolve by becoming smarter and by leveraging cutting-edge technologies. Over the last three decades, we witnessed a tremendous growth of computer vision knowledge through research in academia and industry. More recently, in the last decade, we are finally seeing exciting applications of computer vision. Computer vision plays a fundamental role in the advanced driver assistance systems (ADAS), a field which is of particular interest to the evolution of transportation systems. For example, forward-facing driver assistance functions (such as road sign detection, lane departure warning, and autonomous emergency braking) are heavily relying on information received from a camera. The systems capture video data at high frame rate and process this information in order to warn the driver that the car is moving faster than the posted speed limit or to tell the driver of an unintentional lane drift. The goal of this chapter is to outline key components of ADAS, show how computer vision fits in the system, and describe its contribution to success of ADAS.

Implementation Considerations for Automotive Vision Systems on a Fixed-Point DSP

In this chapter we evaluate numerical requirements for implementation of camera-based lateral position detection algorithms, such as lane keep assistant (LKA) and lane departure warning (LDW) on a fixed-point DSP. We first present methods that address the challenges and requirements of fixed-point design process. The flow proposed is targeted at converting C/C++ code with floating-point operations into C code with integer operations that can then be fed through the native C compiler for a fixed-point DSP. Advanced code optimization and an implementation by DSP-specific, fixed-point C code generation are introduced. We then demonstrate the conversion flow on tracking example (extended Kalman filter) using synthetically generated data, and we analyze trade-offs for algorithm implementation in fixed-point arithmetic. By using the techniques described in this chapter speed can be increased by a factor of up to 10 compared to floating-point emulation on fixed-point hardware.

Image signal processing for front camera based automated driver assistance system

Advanced driver assistance systems (ADAS) are designed to increase drivers situational awareness and road safety by providing essential information, warning and automatic intervention to reduce the possibility/severity of an accident. Front Camera (FC) based ADAS system uses computer vision based techniques to detect obstacles (e.g. pedestrian, cyclist etc) on road from captured image. Typical Image Signal Processing (ISP) is developed to cater cellphones and Digital Still Cameras (DSC) for purpose of human viewing unlike computer vision algorithms whose purpose is to help driver. The given paper explains typical sensor as well as ISP requirements for FC systems deployed in ADAS along with difference compared to cellphones and DSC. The paper proposes typical ISP pipeline which is tuned for computer vision (CV) application. The proposed solution is different in terms of input and output format & bit-depth, processing needs, actual processing algorithm, safety, temperature and thermal constraints. The proposed ISP pipeline is simulated on PC and can be enabled by means of mix of hardware and software on TIs Driver Assistance (TDA) series of processors.

A scalable heterogeneous multicore architecture for ADAS: Presented at HOT CHIPS: A symposium on high performance chips Flint Center, Cupertino, CA

This article consists of a collection of slides from the authors conference presentation. Agenda items include: highlight challenges of implementing Advanced Driver Assistance Systems (ADAS) in embedded systems; Discuss ADAS system options and compromises; High level overview of TDAx SOC; Mapping ADAS use cases to devices from TDAx SOC families.