Zhanping Xu

New electro-optical mixing and correlating sensor: facilities and applications of the photonic mixer device (PMD)

This paper presents first hardware implementation and investigations of a new electro-optic modulator (EOM), called the Photonic Mixer Device (PMD). The PMD is a semiconductor device combining the characteristics of fast optical sensing and modulation. Arranged to a PMD-matrix it looks like a CCD-matrix but additionally provides the depth information of each image pixel using an appropriately modulated scene illumination. Besides this feature of 3D- imaging by means of time-related correlation the PMD-chip will enable on-chip 2D-image processing by means of spatial correlation. The principle of operation of the PMD, possible technologies for realization, facilities and applications will be described. This new device offers high potential for optical sensory systems due to an amazingly simple and powerful procedure of electro-optical mixing and correlation. Both CCD and CMOS are appropriate technologies. The latter will be treated in detail as a single-element Photonic Mixer Device. Here we propose several architectures--including two quadrant (2Q)-PMDs--with readout and preprocessing circuits for both, the phase resp, time-of-flight values and the pixel intensities. Arranging PMD-pixels to a PMD-line or to a PMD-matrix will provide a new generation of flexible and powerful solid-state 3D- cameras based on time-of-flight. According to the EOM- principle, the proposed 2D-mixer requires no additional optical devices, no broadband electronic amplifiers and mixers. This solid-state PMD-array offers even more unique facilities, e.g., performing high speed spatial light modulation up to the GHz range or optical CDMA- communication. The modulation characteristic of the PMD is an important aspect of this mixer. It is associated with optimizing the layout design for the PMD, which will be presented and discussed in this paper, too. PMD test chips have been realized in CMOS technology. Some simulation and first test results of the chip are also presented. The simulation results prove the operation principle of the PMD and provide us with parameters for an optimized layout design. First experimental results verified the expected operation principle of the test PMD.

New active 3D vision system based on rf-modulation interferometry of incoherent light

Presently there is still a remarkable gap between the requirements and the capabilities of 3D- vision in the field of industrial automation, especially in manufacture integrated 100%-quality control. For these and a lot of other applications like security and traffic control a new extremely fast, precise and flexible 3D-camera concept is presented in this paper. In order to obtain the geometrical 3D information, the whole 3D object or 3D scene is illuminated simultaneously by means of rf-modulated light. This is realized by using optical modulators such as Pockels cells or FTR optical components (FTR: Frustrated Total Reflection). The back scattered light represents the depth information within the local delay of the phase front of the rf-modulated light intensity. If the reflected wave front is mixed again within the whole receiving aperture using the same optical 2D-modulation components and the same rf- frequency, an rf-interference pattern is produced. A CCD camera may be applied to sample these rf-modulation interferograms. In order to reconstruct the 3D-image a minimum of three independent interferograms have to be evaluated. They may be produced either by applying three different rf-phases or three different rf-frequencies. This procedure will be able to deliver up to some tens of high resolution 3D images per second with some hundred thousand voxels (volume elements). Such a remarkable progress can be achieved by means of three key important steps: Firstly by separating the opto-electronic receiver device from real-time requirements by using homodyne mixing of CW-modulated light. Secondly by applying the rf- modulation signal as an optical reference signal to the receiving optical mixer. And thirdly by using a throughout 2D layout of the transmitted illumination, further, of the optical mixer in the receiving aperture, and of the optoelectronic sensing element, e.g., a CCD-chip.

New optical four-quadrant phase-detector integrated into a photogate array for small and precise 3D-cameras

The photonic mixer device (PMD) is a new electro-optical mixing semiconductor device. Integrated into a line or an array it may contribute a significant improvement in developing an extremely fast, flexible, robust and low cost 3D-solid-state camera. Three dimensional (3D)-cameras are of dramatically increasing interest in industrial automation, especially for production integrated quality control, in- house navigation, etc. The type of 3D-camera here under consideration is based on the principle of time-of-flight respectively phase delay of surface reflected echoes of rf- modulated light. In contrast to 3D-laser radars there is no scanner required since the whole 3D-scene is illuminated simultaneously using intensity-modulated incoherent light, e.g. in the 10 to 1000 MHz range. The rf-modulated light reflected from the 3D-scene represents the total depth information within the local delay of the back scattered phase front. If this incoming wave front is again rf- modulated by a 2D-mixer within the whole receiving aperture we get a quasi-stationary rf-inference pattern respectively rf-interferogram which may be captured by means of a conventional CCD-camera. This procedure is called rf- modulation interferometry (RFMI). Corresponding to first simulative results the new PMD-array will be appropriate to the RFMI-procedure. Though looking like a modified CCD-array or CMOS-photodetector array it will be able to perform both, the pixelwise mixing process for phase delay respectively depth evaluation and the pixelwise light intensity acquisition for gray level or color evaluation. Further advantageous properties are achieved by means of a four- quadrant (4Q)-PMD array which operates as a balanced inphase/quadrature phase (I/Q)-mixer and will be able to capture the total 3D-scene information of several 100,000 voxels within the microsecond(s) - to ms-range.

New Powerful Sensory Tool in Automotive Safety Systems Based on PMD-Technology

The application of electronic imaging for driver support and occupant safety is actually one of the most challenging tasks in automotive research activities. Most efforts aim to apply two-dimension al imagers in CCD-, CMOS- and TFA-technology so far. Now a further imaging technology has been developed : the PMD-technology, which directly enables 3D-imaging by means of the time-offlight (TOF) principle . This approach offers an extremel y high potential for new solutions in this application field . The PMD-Technology opens up amazing new perspectives for occupant detection systems as well as the observation of vehicles outer area (see Fig . 1.).

Three-dimensional intelligent sensing based on the PMD technology

With this paper we present new 3D sensing technique based on the novel Photonic Mixer Device (PMD), a new generation of smart 3D sensor, which provides a brilliant interface between the world of incoherent light and the world of electronic signal processing. As a new semiconductor device, it combines fast optical sensing and mixing in one component of pixel size by its unique and powerful principle of operation. Based on standard CMOS-technology, it can be easily integrated into PMD sensing arrays, providing both 3D depth and intensity information of the scene. The presented 3D TOF ranging system based on PMD measures the phase and time delay of the back scattered optical signal. The RF- modulated light reflected from the 3D-scene represents the total 3D depth information within the aperture of the PMD receiver. Since the whole 3D-scene is illuminated simultaneously by using intensity-modulated light, the PMD- array on the receiver side performs parallel electro-optical mixing and correlation and delivers an optimal evaluation of time-of-flight and the optical power for each PMD pixel. So there is no scanner required in contrast to the conventional 3D-laser radar systems. The introduction of the PMD into the 3D range sensing technique offers very attractive solutions for the realization of flexible, extremely fast and robust low-cost 3D solid-state smart ranging systems.

Electro-optical correlation arrangement for fast 3D cameras: properties and facilities of the electro-optical mixer device

For advanced performances related to 3D metrology in production, security and traffic control, the Institut fur Nachrichtenverarbeitung is currently engaged in the investigation of a new 3D-camera system based on rf- modulation interferometry, other than triangulation. The new 3D-camera system proposed in this paper uses the techniques of optical rf-modulation of the active illumination source and the correlation of these signals reflected from a target within a large aperture. This method yields all phase- correlation functions, relating to each voxel of the 3D- scene in parallel. Hence, the system delivers a fast parallel measurement and evaluation process by completely using a 2D transmitting and receiving path. To fulfill this concept, we have set up a binocular system using a modified Pockels-cell arrangement and a modulated laser transmitter. Since the electro-optical modulator plays a key important role in realizing the whole system, we would discuss the modified arrangement of stacked birefringent KD*P crystals that incorporates the properties of large aperture and high frequency modulation in the region of some tenth of megahertz. Additionally, an optical reference path is introduced to compensate for the fluctuation of the light source, thus enhancing the capability of the system. The experimental results of this reformed system will be presented in this paper, e.g., the standard deviation of (lambda) /2500 being obtained.

Enhanced multiprobing recovering algorithm based on color mixed nonlinear modulation and its application in a 3D vision system

The paper introduces a fast and enhanced recovering algorithm and its application in an active 3D-color vision system. The algorithm is based on the processing of several non-linearly modulated optical test signals of different colors. The processing approach arises from minimizing errors caused by using non-linear modulators in an active vision system, i.e., recovering 3D properties from higher order terms of a Fourier series expansion of the non- linear modulation. Two aspects are worth mentioning: Firstly, the modulation depth of non- linear optical components such as Pockels cells can be exploited much beyond the linear region if assisted by the appropriate recovering algorithm and thus increasing the effective aperture of the optical system. Secondly, the same algorithm can be adaptive to a synthetic non-linear modulation, i.e., the various incoherent signals used as color probes are synchronously modulated each with different characterizing rf signals by means of corresponding optical modulators. These signals are then further incoherently superposed in the transmission medium. After having been reflected from and interacted with the object of interest, the selectively attenuated signals are demodulated using a single modulator. In this process phase, color and other information are simultaneously demodulated. Therefore a single black-white CCD camera may be utilized to sample the 2D-rf interferograms, which are fast and analytically processed by the proposed algorithm in order to extract 3D ranges, colors, and other properties of the interesting object.

Synchronization in spread spectrum laser radar systems based on PMD-DLL

This paper proposes a new optoelectronic delay locked loop (OE-DLL) and its use in optical ranging systems. The so called PMD-DLL receiver module is based on a novel electro-optical modulator (EOM), called the Photonic Mixer Device (PMD). This sensor element is a semiconductor device, which combines fast optical sensing and mixing of incoherent light signals in one component part by its unique and powerful principle of operation. Integration of some simple additional on-chip components offers a high integrated electro-optical correlation unit. Simulations and experimental results have already impressively verified the operation principle of PMD structures, all realized in CMOS technology so far. Although other technologies are also promising candidates for the PMD realization they should not be further discussed in this contribution. The principle of the new DLL approach is intensively discussed in this paper. Theoretical analysis as well as experimental results of a realized PMD-DLL system are demonstrated and judged. Due to the operation principle of sophisticated PMD devices and their unique features, a correlation process may be realized in order to synchronize a reflected incoherent light wave with an electronic reference signal. The phase shift between both signals represents the distance to an obstacle and may be determined by means of the synchronization process. This new approach, avoiding so far needed critical components such as broadband amplifiers and mixers for the detection of small photo currents in optical distance measurement, offers an extremely fast and precise phase determination in ranging applications based on the time- of-flight (TOF) principle. However, the optical measurement signal may be incoherent -- therefore a laser source is not needed imperatively. The kind of waveform used for the modulation of the light signal is variable and depends on the demands of every specific application. Even if there are plenty other alternatives (e.g., heterodyne techniques), in this contribution only so called quasi-heterodyne techniques - - also known as phase shifting methods -- are discussed and used for the implementation. The light modulation schemes described in this contribution are square-wave as well as pseudo-noise modulation. The latter approach, inspired by the wide spread use in communication as well as in position detection (e.g., IS-95 and GPS), offers essential advantages and is the most promising modulation method for the ranging approach. So called CDMA (code division multiple access) systems form a major task in communication technology investigations since the third generation mobile phone standard is also partly based on this principle. Fast and reliable synchronization in direct sequence spread spectrum communication systems (DSSS) differs hardly from the already mentioned ranging approach and will also be discussed. The possibility to integrate all components in a monolithic PMD based DLL design is also presented and discussed. This method might offer the feature to integrate complete lines or matrixes of PMD based DLLs for highly parallel, multidimensional ranging. Finally, an outlook is given with regard to further optimized PMD front ends. An estimation of the expected characteristics concerning accuracy and speed of the distance measurement is given in conclusion.

Power transmittance of optimized aspherical lens with large numerical aperture

An optimized aspherical lens with large numerical aperture is demanded in many optical systems. In addressing the requirements, we have obtained analytical solutions of such a desired two-side aspherical lens by solving differential equations. Therefore geometrical parameters and index of refraction of the lens are explicitly expressed in the formulas. Lots of aspects of the aspherical lens can be seen by analyzing the obtained formulas rather than conventionally analyzing discrete data. Criteria of manufacturing the optimized lenses in different precision form uniquely depend on the refractive index of the plastics or glass material. On the basis of summarizing the work in this paper, we further derive formulas of power transmission for the optimized aspherical lens in several cases. The power transmission analysis is associated with typical radiation distribution of a source such as Lambertian or Gaussian distribution. Two polarized states of optical waves are also taken into account. Another criterion of manufacturing aspherical lenses with a specific transmittance is given according to the numerical aperture. It can be used to evaluate the power transmittance of plastic aspherical lenses without anti-reflection coating. A design according to the proper criteria allows more tolerances in the process of plastic lens manufacturing and assembling, while the analytically optimized results are more adaptive to compensation design for correcting deformation of materials.

New optical 2D modulator jacketed in rotational plastic optics

Optical and therefore nontactile 3D-measurement techniques are of increasing interest in industrial automation, especially in quality control and guidance of automotive vehicles. In connection with these demands, a new type of optical modulator jacketed in rotational plastic optics is introduced in the paper. Furthermore first results obtained by simulation studies will be presented. A simple nevertheless effective way of obtaining 3D information is to illuminate the whole 3D object or scene simultaneously with rf-modulated light. This can be well achieved by using the suggested optical modulator that incorporates the properties of a high aperture and minimum aberration in the 3D-imaging process. The mentioned modulator makes use of the effect of Frustrated Total Reflection (FTR). To exploit this FTR effect in an optical 2D mixer, the gap width between media of higher dense has to be modulated by an rf-voltage applied to a piezo crystal as an rf-controlled tuning medium. Considering the limited modulation bandwidth due to the parasitic capacity of the piezo crystal, the geometrical dimension of the modulator must be made as small as possible. Therefore the spot of the light is collimated at the focal point of the jacketing rotational ellipsoid. The integrated component made of plastic optics and piezo crystal plays a substantial role for the optical modulation and imaging. Some simulation results of this optical device show that the inherent non-linearity of the FTR modulator may be neglected in practical applications, thus yielding a high modulation depth. Furthermore, a 3D-image system adopting this plastic-made optics is also depicted in the paper, which is robust and handy for several industrial applications.