Buyue Zhang

A Surround View Camera Solution for Embedded Systems

Automotive surround view camera system is an emerging automotive ADAS (Advanced Driver Assistance System) technology that assists the driver in parking the vehicle safely by allowing him/her to see a top-down view of the 360 degree surroundings of the vehicle. Such a system normally consists of four to six wide-angle (fish-eye lens) cameras mounted around the vehicle, each facing a different direction. From these camera inputs, a composite bird-eye view of the vehicle is synthesized and shown to the driver in real-time during parking. In this paper, we present a surround view camera solution that consists of three key algorithm components: geometric alignment, photometric alignment, and composite view synthesis. Our solution produces a seamlessly stitched bird-eye view of the vehicle from four cameras. It runs real-time on DSP C66x producing an 880x1080 output video at 30 fps.

A real-time auto white balance algorithm for mobile phone cameras

Auto white balance (AWB), a critical component of the image pipeline in digital cameras, is responsible for producing accurate color by automatically removing the undersired color cast introduced by the illumination. AWB estimates the scene illumination by processing the digital values of the pixels in the captured image; therefore, objects colors in the scene are often confused with the color of the light source, leading to wrong color cast in white balanced images. In this paper, we propose a color histogram based AWB algorithm that is capable of producing accurate color in the presence of dominant object colors. Our approach is based on a statistical estimation of the probabilities of colors in natural scenes under different illuminations. These statistics can be easily collected during the AWB calibration phase using standard equipment. The proposed AWB algorithm is computationally efficient and runs real-time in a mobile phone camera.

Embedded stereoscopic 3D camera system for mobile platforms

In this paper, we describe an embedded programmable stereoscopic 3D camera system for mobile platforms. With the proliferation of stereoscopic televisions, computer monitors, and even auto-stereoscopic LCDs, as well as a growing inventory of stereoscopic games and movies, there is a growing demand for stereoscopic 3D content creation. We present each of the key components that enable the stereoscopic 3D video capture and playback on the embedded platform, including imaging, codec, graphics, and display subsystems. We describe several unique and new 3D image processing algorithms and explain how they are integrated into the stereo 3D system.

Auto convergence for stereoscopic 3D cameras

Viewing comfort is an important concern for 3-D capable consumer electronics such as 3-D cameras and TVs. Consumer generated content is typically viewed at a close distance which makes the vergence-accommodation conflict particularly pronounced, causing discomfort and eye fatigue. In this paper, we present a Stereo Auto Convergence (SAC) algorithm for consumer 3-D cameras that reduces the vergence-accommodation conflict on the 3-D display by adjusting the depth of the scene automatically. Our algorithm processes stereo video in realtime and shifts each stereo frame horizontally by an appropriate amount to converge on the chosen object in that frame. The algorithm starts by estimating disparities between the left and right image pairs using correlations of the vertical projections of the image data. The estimated disparities are then analyzed by the algorithm to select a point of convergence. The current and target disparities of the chosen convergence point determines how much horizontal shift is needed. A disparity safety check is then performed to determine whether or not the maximum and minimum disparity limits would be exceeded after auto convergence. If the limits would be exceeded, further adjustments are made to satisfy the safety limits. Finally, desired convergence is achieved by shifting the left and the right frames accordingly. Our algorithm runs real-time at 30 fps on a TI OMAP4 processor. It is tested using an OMAP4 embedded prototype stereo 3-D camera. It significantly improves 3-D viewing comfort.

A real-time misalignment correction algorithm for stereoscopic 3D cameras

Camera calibration is an important problem for stereo 3-D cameras since the misalignment between the two views can lead to vertical disparities that significantly degrade 3-D viewing quality. Offline calibration during manufacturing is not always an option especially for mass produced cameras due to cost. In addition, even if one-time calibration is performed during manufacturing, its accuracy cannot be maintained indefinitely because environmental factors can lead to changes in camera hardware. In this paper, we propose a real-time stereo calibration solution that runs inside a consumer camera and continuously estimates and corrects for the misalignment between the stereo cameras. Our algorithm works by processing images of natural scenes and does not require the use of special calibration charts. The algorithm first estimates the disparity in horizontal and vertical directions between the corresponding blocks from stereo images. Then, this initial estimate is refined with two dimensional search using smaller sub-blocks. The displacement data and block coordinates are fed to a modified affine transformation model and outliers are discarded to keep the modeling error low. Finally, the estimated affine parameters are split by half and misalignment correction is applied to each view accordingly. The proposed algorithm significantly reduces the misalignment between stereo frames and enables a more comfortable 3-D viewing experience.