Thorsten Ringbeck

New insights into the calibration of ToF-sensors

Time-of-flight (ToF) sensors have become an alternative to conventional distance sensing techniques like laser scanners or image based stereo. ToF sensors provide full range distance information at high frame-rates and thus have a significant impact onto current research in areas like online object recognition, collision prevention or scene reconstruction. However, ToF cameras like the photonic mixer device (PMD) still exhibit a number of challenges regarding static and dynamic effects, e.g. systematic distance errors and motion artefacts, respectively. Sensor calibration techniques reducing static system errors have been proposed and show promising results. However, current calibration techniques in general need a large set of reference data in order to determine the corresponding parameters for the calibration model. This paper introduces a new calibration approach which combines different demodulation techniques for the ToF- camera s reference signal. Examples show, that the resulting combined demodulation technique yields improved distance values based on only two required reference data sets.

New Powerful Sensory Tool in Automotive Safety Systems Based on PMD-Technology

The application of electronic imaging for driver support and occupant safety is actually one of the most challenging tasks in automotive research activities. Most efforts aim to apply two-dimension al imagers in CCD-, CMOS- and TFA-technology so far. Now a further imaging technology has been developed : the PMD-technology, which directly enables 3D-imaging by means of the time-offlight (TOF) principle . This approach offers an extremel y high potential for new solutions in this application field . The PMD-Technology opens up amazing new perspectives for occupant detection systems as well as the observation of vehicles outer area (see Fig . 1.).

Three-dimensional intelligent sensing based on the PMD technology

With this paper we present new 3D sensing technique based on the novel Photonic Mixer Device (PMD), a new generation of smart 3D sensor, which provides a brilliant interface between the world of incoherent light and the world of electronic signal processing. As a new semiconductor device, it combines fast optical sensing and mixing in one component of pixel size by its unique and powerful principle of operation. Based on standard CMOS-technology, it can be easily integrated into PMD sensing arrays, providing both 3D depth and intensity information of the scene. The presented 3D TOF ranging system based on PMD measures the phase and time delay of the back scattered optical signal. The RF- modulated light reflected from the 3D-scene represents the total 3D depth information within the aperture of the PMD receiver. Since the whole 3D-scene is illuminated simultaneously by using intensity-modulated light, the PMD- array on the receiver side performs parallel electro-optical mixing and correlation and delivers an optimal evaluation of time-of-flight and the optical power for each PMD pixel. So there is no scanner required in contrast to the conventional 3D-laser radar systems. The introduction of the PMD into the 3D range sensing technique offers very attractive solutions for the realization of flexible, extremely fast and robust low-cost 3D solid-state smart ranging systems.

PMD-PLL: receiver structure for incoherent communication and ranging systems

This paper proposes a high integrated optoelectronic phase locked loop for the use in optical data transmission as well as in optical ranging systems. The so called PMD-PLL receiver module is based on a novel electro-optical modulator, called the Photonic Mixer Device (PMD). The sensor is a semiconductor device, which combines fast optical sensing and modulation of incoherent light signals in one component part by its unique and powerful principle of operation. Simulations and experimental results have already verified the operation principle of PMD structures, in CMOS-technology so far. But also other technologies may be suitable for the PMD realization.

MSM-PMD as correlation receiver in a new 3D-ranging system

This contribution describes a ranging system based on a new kind of optical sensor, called the Photonic Mixer Device (PMD). This unit combines fast optical sensing and modulation of incoherent light signals in one component part by its unique and powerful principle of operation. The key feature of this new sensor is the direct detection of the correlation value between the light and an electrical reference signal as an analog voltage value. The realized ranging system is based on the following concept: a laser diode or LED is modulated with a PN-code sequence and the reference signal consists of the same PN-code sequence. With a delay unit the phase of the reference signal is shifted until the PMD-sensor returns an expected correlation value. The delay between the reference and the reflected light signal implies information concerning the distance to the reflecting object. An array of these sensors provides a 3D- ranging system that could be used for, e.g., collision avoidance in new traffic control systems or in product monitoring systems. Fast modulation of the laser and the PMD provides a high distance resolution. Nowadays PMD-realizations are based on standard CMOS technology, but we also keep investigation in a new kind of PMD based on MSM technology (metal- semiconductor-metal).

Synchronization in spread spectrum laser radar systems based on PMD-DLL

This paper proposes a new optoelectronic delay locked loop (OE-DLL) and its use in optical ranging systems. The so called PMD-DLL receiver module is based on a novel electro-optical modulator (EOM), called the Photonic Mixer Device (PMD). This sensor element is a semiconductor device, which combines fast optical sensing and mixing of incoherent light signals in one component part by its unique and powerful principle of operation. Integration of some simple additional on-chip components offers a high integrated electro-optical correlation unit. Simulations and experimental results have already impressively verified the operation principle of PMD structures, all realized in CMOS technology so far. Although other technologies are also promising candidates for the PMD realization they should not be further discussed in this contribution. The principle of the new DLL approach is intensively discussed in this paper. Theoretical analysis as well as experimental results of a realized PMD-DLL system are demonstrated and judged. Due to the operation principle of sophisticated PMD devices and their unique features, a correlation process may be realized in order to synchronize a reflected incoherent light wave with an electronic reference signal. The phase shift between both signals represents the distance to an obstacle and may be determined by means of the synchronization process. This new approach, avoiding so far needed critical components such as broadband amplifiers and mixers for the detection of small photo currents in optical distance measurement, offers an extremely fast and precise phase determination in ranging applications based on the time- of-flight (TOF) principle. However, the optical measurement signal may be incoherent -- therefore a laser source is not needed imperatively. The kind of waveform used for the modulation of the light signal is variable and depends on the demands of every specific application. Even if there are plenty other alternatives (e.g., heterodyne techniques), in this contribution only so called quasi-heterodyne techniques - - also known as phase shifting methods -- are discussed and used for the implementation. The light modulation schemes described in this contribution are square-wave as well as pseudo-noise modulation. The latter approach, inspired by the wide spread use in communication as well as in position detection (e.g., IS-95 and GPS), offers essential advantages and is the most promising modulation method for the ranging approach. So called CDMA (code division multiple access) systems form a major task in communication technology investigations since the third generation mobile phone standard is also partly based on this principle. Fast and reliable synchronization in direct sequence spread spectrum communication systems (DSSS) differs hardly from the already mentioned ranging approach and will also be discussed. The possibility to integrate all components in a monolithic PMD based DLL design is also presented and discussed. This method might offer the feature to integrate complete lines or matrixes of PMD based DLLs for highly parallel, multidimensional ranging. Finally, an outlook is given with regard to further optimized PMD front ends. An estimation of the expected characteristics concerning accuracy and speed of the distance measurement is given in conclusion.

Phase-lock techniques using new correlation receivers for TOF-laser ranging

Optical distance measurement, based on the time-of-flight (TOF) principle, suffer from the unfavorable ratio of desired accuracy and the small received signal power, which is reflected by the target. Many receiver concepts are discussed in the past, but only any concepts are putted into practice. The concept of Photonic Mixer Devices (PMD) offers any interesting features for laser ranging systems, like fast photo-detection with inherent mixing, accumulation of a adequate signal power by a following integration, flicker noise suppression, background illumination, and interference rejection of two-channel designs. The combination of PMD and phase-lock techniques additionally enables a continuous tracking of the object distance, his relative velocity to the measurement system, and the restoration of the received optical signal. Different design concepts of the Photonic Mixer Device (PMD) with their characteristics and advantages as the key component in an Electro-Optical Phase-Locked Loop (OE-PLL) and an Electro-Optical Delay-Locked Loop (OE-DLL) will be discussed.

Optical GMSK modem for infrared wireless communication based on a new receiver principle in CMOS technology

The PMD-technology opens up a wide area of solutions for all detector problems where a high phase accuracy is required. One important application of the new Photonic Mixer Device is optical distance measurement based on the time-of-flight principle. The high integration of PMD smart-pixels in an array based on CMOS-technology means a breakthrough in optical 3D-imaging. Additionally to the inherent mixing feature, even more functionality could be integrated according to the well-known active pixel sensor concept.

3D Time-of-Flight (ToF)

Trotz steigender Verkehrsdichte ist die Zahl der Verkehrsunfalle mit Personenschaden in den letzten Jahren gesunken. Um zukunftige Fahrzeuge sowohl fur die Insassen als auch fur andere Verkehrsteilnehmer noch sicherer zu machen, wird eine zunehmend dreidimensionale Umfelderfassung durch das Fahrzeug notwendig. Eine entsprechende 3D-Sensorik ist in der Lage, gefahrliche Situationen vorausschauend zu erkennen, den Fahrer bestmoglich zu unterstutzen und somit Unfalle zu vermeiden. Aber auch im Falle eines nicht mehr zu vermeidenden Unfalls lasst sich das Verletzungsrisiko fur alle Beteiligten minimieren.

Touchless Detailed 3D Scan of Human Hand Anatomy Using Time-of-Flight Cameras

Time-of-Flight (ToF) cameras provide an easy, robust and touchless way to accomplish a three-dimensional surface scan at real-time framerates of ≥25Hz and with lateral resolutions of up to 204x204 pixels. This paper describes a processing pipeline to detect anatomical details the hand like finger tips, root finger joints and center of palm. The processing pipeline robustly analyzes hand in the field of view even if they are covered by gloves or sleeves. Additionally, the available depth data leads to a robust segmentation and validation of the anatomical structures.