Bernd Buxbaum

New electro-optical mixing and correlating sensor: facilities and applications of the photonic mixer device (PMD)

This paper presents first hardware implementation and investigations of a new electro-optic modulator (EOM), called the Photonic Mixer Device (PMD). The PMD is a semiconductor device combining the characteristics of fast optical sensing and modulation. Arranged to a PMD-matrix it looks like a CCD-matrix but additionally provides the depth information of each image pixel using an appropriately modulated scene illumination. Besides this feature of 3D- imaging by means of time-related correlation the PMD-chip will enable on-chip 2D-image processing by means of spatial correlation. The principle of operation of the PMD, possible technologies for realization, facilities and applications will be described. This new device offers high potential for optical sensory systems due to an amazingly simple and powerful procedure of electro-optical mixing and correlation. Both CCD and CMOS are appropriate technologies. The latter will be treated in detail as a single-element Photonic Mixer Device. Here we propose several architectures--including two quadrant (2Q)-PMDs--with readout and preprocessing circuits for both, the phase resp, time-of-flight values and the pixel intensities. Arranging PMD-pixels to a PMD-line or to a PMD-matrix will provide a new generation of flexible and powerful solid-state 3D- cameras based on time-of-flight. According to the EOM- principle, the proposed 2D-mixer requires no additional optical devices, no broadband electronic amplifiers and mixers. This solid-state PMD-array offers even more unique facilities, e.g., performing high speed spatial light modulation up to the GHz range or optical CDMA- communication. The modulation characteristic of the PMD is an important aspect of this mixer. It is associated with optimizing the layout design for the PMD, which will be presented and discussed in this paper, too. PMD test chips have been realized in CMOS technology. Some simulation and first test results of the chip are also presented. The simulation results prove the operation principle of the PMD and provide us with parameters for an optimized layout design. First experimental results verified the expected operation principle of the test PMD.

New optical four-quadrant phase-detector integrated into a photogate array for small and precise 3D-cameras

The photonic mixer device (PMD) is a new electro-optical mixing semiconductor device. Integrated into a line or an array it may contribute a significant improvement in developing an extremely fast, flexible, robust and low cost 3D-solid-state camera. Three dimensional (3D)-cameras are of dramatically increasing interest in industrial automation, especially for production integrated quality control, in- house navigation, etc. The type of 3D-camera here under consideration is based on the principle of time-of-flight respectively phase delay of surface reflected echoes of rf- modulated light. In contrast to 3D-laser radars there is no scanner required since the whole 3D-scene is illuminated simultaneously using intensity-modulated incoherent light, e.g. in the 10 to 1000 MHz range. The rf-modulated light reflected from the 3D-scene represents the total depth information within the local delay of the back scattered phase front. If this incoming wave front is again rf- modulated by a 2D-mixer within the whole receiving aperture we get a quasi-stationary rf-inference pattern respectively rf-interferogram which may be captured by means of a conventional CCD-camera. This procedure is called rf- modulation interferometry (RFMI). Corresponding to first simulative results the new PMD-array will be appropriate to the RFMI-procedure. Though looking like a modified CCD-array or CMOS-photodetector array it will be able to perform both, the pixelwise mixing process for phase delay respectively depth evaluation and the pixelwise light intensity acquisition for gray level or color evaluation. Further advantageous properties are achieved by means of a four- quadrant (4Q)-PMD array which operates as a balanced inphase/quadrature phase (I/Q)-mixer and will be able to capture the total 3D-scene information of several 100,000 voxels within the microsecond(s) - to ms-range.

New Powerful Sensory Tool in Automotive Safety Systems Based on PMD-Technology

The application of electronic imaging for driver support and occupant safety is actually one of the most challenging tasks in automotive research activities. Most efforts aim to apply two-dimension al imagers in CCD-, CMOS- and TFA-technology so far. Now a further imaging technology has been developed : the PMD-technology, which directly enables 3D-imaging by means of the time-offlight (TOF) principle . This approach offers an extremel y high potential for new solutions in this application field . The PMD-Technology opens up amazing new perspectives for occupant detection systems as well as the observation of vehicles outer area (see Fig . 1.).

Three-dimensional intelligent sensing based on the PMD technology

With this paper we present new 3D sensing technique based on the novel Photonic Mixer Device (PMD), a new generation of smart 3D sensor, which provides a brilliant interface between the world of incoherent light and the world of electronic signal processing. As a new semiconductor device, it combines fast optical sensing and mixing in one component of pixel size by its unique and powerful principle of operation. Based on standard CMOS-technology, it can be easily integrated into PMD sensing arrays, providing both 3D depth and intensity information of the scene. The presented 3D TOF ranging system based on PMD measures the phase and time delay of the back scattered optical signal. The RF- modulated light reflected from the 3D-scene represents the total 3D depth information within the aperture of the PMD receiver. Since the whole 3D-scene is illuminated simultaneously by using intensity-modulated light, the PMD- array on the receiver side performs parallel electro-optical mixing and correlation and delivers an optimal evaluation of time-of-flight and the optical power for each PMD pixel. So there is no scanner required in contrast to the conventional 3D-laser radar systems. The introduction of the PMD into the 3D range sensing technique offers very attractive solutions for the realization of flexible, extremely fast and robust low-cost 3D solid-state smart ranging systems.

PMD-PLL: receiver structure for incoherent communication and ranging systems

This paper proposes a high integrated optoelectronic phase locked loop for the use in optical data transmission as well as in optical ranging systems. The so called PMD-PLL receiver module is based on a novel electro-optical modulator, called the Photonic Mixer Device (PMD). The sensor is a semiconductor device, which combines fast optical sensing and modulation of incoherent light signals in one component part by its unique and powerful principle of operation. Simulations and experimental results have already verified the operation principle of PMD structures, in CMOS-technology so far. But also other technologies may be suitable for the PMD realization.

MSM-PMD as correlation receiver in a new 3D-ranging system

This contribution describes a ranging system based on a new kind of optical sensor, called the Photonic Mixer Device (PMD). This unit combines fast optical sensing and modulation of incoherent light signals in one component part by its unique and powerful principle of operation. The key feature of this new sensor is the direct detection of the correlation value between the light and an electrical reference signal as an analog voltage value. The realized ranging system is based on the following concept: a laser diode or LED is modulated with a PN-code sequence and the reference signal consists of the same PN-code sequence. With a delay unit the phase of the reference signal is shifted until the PMD-sensor returns an expected correlation value. The delay between the reference and the reflected light signal implies information concerning the distance to the reflecting object. An array of these sensors provides a 3D- ranging system that could be used for, e.g., collision avoidance in new traffic control systems or in product monitoring systems. Fast modulation of the laser and the PMD provides a high distance resolution. Nowadays PMD-realizations are based on standard CMOS technology, but we also keep investigation in a new kind of PMD based on MSM technology (metal- semiconductor-metal).

Synchronization in spread spectrum laser radar systems based on PMD-DLL

This paper proposes a new optoelectronic delay locked loop (OE-DLL) and its use in optical ranging systems. The so called PMD-DLL receiver module is based on a novel electro-optical modulator (EOM), called the Photonic Mixer Device (PMD). This sensor element is a semiconductor device, which combines fast optical sensing and mixing of incoherent light signals in one component part by its unique and powerful principle of operation. Integration of some simple additional on-chip components offers a high integrated electro-optical correlation unit. Simulations and experimental results have already impressively verified the operation principle of PMD structures, all realized in CMOS technology so far. Although other technologies are also promising candidates for the PMD realization they should not be further discussed in this contribution. The principle of the new DLL approach is intensively discussed in this paper. Theoretical analysis as well as experimental results of a realized PMD-DLL system are demonstrated and judged. Due to the operation principle of sophisticated PMD devices and their unique features, a correlation process may be realized in order to synchronize a reflected incoherent light wave with an electronic reference signal. The phase shift between both signals represents the distance to an obstacle and may be determined by means of the synchronization process. This new approach, avoiding so far needed critical components such as broadband amplifiers and mixers for the detection of small photo currents in optical distance measurement, offers an extremely fast and precise phase determination in ranging applications based on the time- of-flight (TOF) principle. However, the optical measurement signal may be incoherent -- therefore a laser source is not needed imperatively. The kind of waveform used for the modulation of the light signal is variable and depends on the demands of every specific application. Even if there are plenty other alternatives (e.g., heterodyne techniques), in this contribution only so called quasi-heterodyne techniques - - also known as phase shifting methods -- are discussed and used for the implementation. The light modulation schemes described in this contribution are square-wave as well as pseudo-noise modulation. The latter approach, inspired by the wide spread use in communication as well as in position detection (e.g., IS-95 and GPS), offers essential advantages and is the most promising modulation method for the ranging approach. So called CDMA (code division multiple access) systems form a major task in communication technology investigations since the third generation mobile phone standard is also partly based on this principle. Fast and reliable synchronization in direct sequence spread spectrum communication systems (DSSS) differs hardly from the already mentioned ranging approach and will also be discussed. The possibility to integrate all components in a monolithic PMD based DLL design is also presented and discussed. This method might offer the feature to integrate complete lines or matrixes of PMD based DLLs for highly parallel, multidimensional ranging. Finally, an outlook is given with regard to further optimized PMD front ends. An estimation of the expected characteristics concerning accuracy and speed of the distance measurement is given in conclusion.

Optical GMSK modem for infrared wireless communication based on a new receiver principle in CMOS technology

The PMD-technology opens up a wide area of solutions for all detector problems where a high phase accuracy is required. One important application of the new Photonic Mixer Device is optical distance measurement based on the time-of-flight principle. The high integration of PMD smart-pixels in an array based on CMOS-technology means a breakthrough in optical 3D-imaging. Additionally to the inherent mixing feature, even more functionality could be integrated according to the well-known active pixel sensor concept.

3D Time-of-Flight (ToF)

Trotz steigender Verkehrsdichte ist die Zahl der Verkehrsunfalle mit Personenschaden in den letzten Jahren gesunken. Um zukunftige Fahrzeuge sowohl fur die Insassen als auch fur andere Verkehrsteilnehmer noch sicherer zu machen, wird eine zunehmend dreidimensionale Umfelderfassung durch das Fahrzeug notwendig. Eine entsprechende 3D-Sensorik ist in der Lage, gefahrliche Situationen vorausschauend zu erkennen, den Fahrer bestmoglich zu unterstutzen und somit Unfalle zu vermeiden. Aber auch im Falle eines nicht mehr zu vermeidenden Unfalls lasst sich das Verletzungsrisiko fur alle Beteiligten minimieren.

Touchless Detailed 3D Scan of Human Hand Anatomy Using Time-of-Flight Cameras

Time-of-Flight (ToF) cameras provide an easy, robust and touchless way to accomplish a three-dimensional surface scan at real-time framerates of ≥25Hz and with lateral resolutions of up to 204x204 pixels. This paper describes a processing pipeline to detect anatomical details the hand like finger tips, root finger joints and center of palm. The processing pipeline robustly analyzes hand in the field of view even if they are covered by gloves or sleeves. Additionally, the available depth data leads to a robust segmentation and validation of the anatomical structures.