Milivoje Aleksic

3D image processing architecture for camera phones

Putting high quality and easy-to-use 3D technology into the hands of regular consumers has become a recent challenge as interest in 3D technology has grown. Making 3D technology appealing to the average user requires that it be made fully automatic and foolproof. Designing a fully automatic 3D capture and display system requires: 1) identifying critical 3D technology issues like camera positioning, disparity control rationale, and screen geometry dependency, 2) designing methodology to automatically control them. Implementing 3D capture functionality on phone cameras necessitates designing algorithms to fit within the processing capabilities of the device. Various constraints like sensor position tolerances, sensor 3A tolerances, post-processing, 3D video resolution and frame rate should be carefully considered for their influence on 3D experience. Issues with migrating functions such as zoom and pan from the 2D usage model (both during capture and display) to 3D needs to be resolved to insure the highest level of user experience. It is also very important that the 3D usage scenario (including interactions between the user and the capture/display device) is carefully considered. Finally, both the processing power of the device and the practicality of the scheme needs to be taken into account while designing the calibration and processing methodology.

R/G/B color crosstalk characterization and calibration for LCD displays

LCD displays exhibit significant color crosstalks between their red, green and blue channels (more or less depending on the type of LCD technology). This problem, if it is not addressed properly, leads to (a) a significant color errors in the rendered images on LCD displays and (b) a significant gray tracking problem. The traditional method for addressing this problem has been to use a 3x3 color correction matrix in the display processing pipe. Experimental data clearly shows that this linear model for color correction is not sufficient to address color crosstalk problem in LCD displays. Herein, it is proposed to use higher order polynomials for color correction in the display processing pipe. This paper presents detailed experimental results and comparative analysis on using polynomial models with different orders for color correction.

An efficient spectral-based calibration method for RGB white-balancing gains under various illumination conditions for cell-phone cameras

Significant sensitivity variations among cell-phone camera modules have been observed. As a result, for an effective and reliable white balancing, per-module RGB-ratios calibration/estimation under various illumination conditions is required. Herein, a new technique is proposed which minimizes/simplifies RGB-ratios calibration/estimation process. The proposed method could be based on either direct image capture or spectral numerical processing--the latter is shown to be more flexible and accurate.

Challenges in Display Color Management (DCM) for Mobile Devices

Systematic and effective color management for displays on mobile devices is becoming increasingly more challenging. A list of the main challenges includes (a) significant differences in display technologies, (b) significant display color response and tone response variability, (c) significant display color gamut variability, (d) significant content gamut variability, (e) mixing content with different color gamuts, (f) Significant variability in viewing conditions and (g) significant mobile display power consumption. This paper will provide general descriptions of the above challenges, their characteristics and complexities.

Comparative performance analysis of two picture adjustment methods: HSV vs. YCbCr

Picture adjustment is referred to those adjustments that affect the four main subjective perceptual image attributes: Hue, Saturation, Brightness (sometimes called Intensity) and Contrast--HSIC adjustments. The common method used for this type of adjustments in a display processing pipe is based on YCbCr color space and a 3x4 color adjustment matrix. Picture adjustments based on this method, however, leads to multiple problems such as adjusting one attribute leads to degradation of other attributes. As an alternative, other color spaces such as HSV can be used to generate more consistent and effective picture adjustments. In this paper, the results of a comparative performance analysis between the two methods based on YCbCr and HSV color spaces (for HSIC adjustments) are presented.

Comparative performance analysis of mobile displays

Cell-phone display performance (in terms of color quality and optical efficiency) has become a critical factor in creating a positive user experience. As a result, there is a significant amount of effort by cell-phone OEMs to provide a more competitive display solution. This effort is focused on using different display technologies (with significantly different color characteristics) and more sophisticated display processors. In this paper, the results of a mobile-display comparative performance analysis are presented. Three cell-phones from major OEMs are selected and their display performances are measured and quantified. Comparative performance analysis is done using display characteristics such as display color gamut size, RGB-channels crosstalk, RGB tone responses, gray tracking performance, color accuracy, and optical efficiency.

Multithreaded real-time 3D image processing software architecture and implementation

Recently, 3D displays and videos have generated a lot of interest in the consumer electronics industry. To make 3D capture and playback popular and practical, a user friendly playback interface is desirable. Towards this end, we built a real time software 3D video player. The 3D video player displays user captured 3D videos, provides for various 3D specific image processing functions and ensures a pleasant viewing experience. Moreover, the player enables user interactivity by providing digital zoom and pan functionalities. This real time 3D player was implemented on the GPU using CUDA and OpenGL. The player provides user interactive 3D video playback. Stereo images are first read by the player from a fast drive and rectified. Further processing of the images determines the optimal convergence point in the 3D scene to reduce eye strain. The rationale for this convergence point selection takes into account scene depth and display geometry. The first step in this processing chain is identifying keypoints by detecting vertical edges within the left image. Regions surrounding reliable keypoints are then located on the right image through the use of block matching. The difference in the positions between the corresponding regions in the left and right images are then used to calculate disparity. The extrema of the disparity histogram gives the scene disparity range. The left and right images are shifted based upon the calculated range, in order to place the desired region of the 3D scene at convergence. All the above computations are performed on one CPU thread which calls CUDA functions. Image upsampling and shifting is performed in response to user zoom and pan. The player also consists of a CPU display thread, which uses OpenGL rendering (quad buffers). This also gathers user input for digital zoom and pan and sends them to the processing thread.

Spectral-based calorimetric calibration of a 3CCD color camera for fast and accurate characterization and calibration of LCD displays

LCD displays exhibit significant amount of variability in their tone-responses, color responses and backlight-modulation responses. LCD display characterization and calibration using a spectrometer or a color meter, however, leads to two basic deficiencies: (a) It can only generate calibration data based on a single spot on the display (usually at panel center); and (b) It generally takes a significant amount of time to do the required measurement. As a result, a fast and efficient system for a full LCD display characterization and calibration is required. Herein, a system based on a 3CCD calorimetrically-calibrated camera is presented which can be used for full characterization and calibration of LCD displays. The camera can provide full tri-stimulus measurements in real time. To achieve high-degree of accuracy, colorimetric calibration of camera is carried out based on spectral method.