John C. Pincenti

Quality and noise measurements in mobile phone video capture

The quality of videos captured with mobile phones has become increasingly important particularly since resolutions and formats have reached a level that rivals the capabilities available in the digital camcorder market, and since many mobile phones now allow direct playback on large HDTVs. The video quality is determined by the combined quality of the individual parts of the imaging system including the image sensor, the digital color processing, and the video compression, each of which has been studied independently. In this work, we study the combined effect of these elements on the overall video quality. We do this by evaluating the capture under various lighting, color processing, and video compression conditions. First, we measure full reference quality metrics between encoder input and the reconstructed sequence, where the encoder input changes with light and color processing modifications. Second, we introduce a system model which includes all elements that affect video quality, including a low light additive noise model, ISP color processing, as well as the video encoder. Our experiments show that in low light conditions and for certain choices of color processing the system level visual quality may not improve when the encoder becomes more capable or the compression ratio is reduced.

Camera motion and mobile imaging

Due to the demanding size and cost constraints of camera phones, the mobile imaging industry needs to address several key challenges in order to achieve the quality of a digital still camera. Minimizing camera-motion introduced image blur is one of them. Film photographers have long used a rule-of-thumb that a hand held 35mm format film camera should have an exposure in seconds that is not longer than the inverse of the focal length in millimeters. Due to the lack of scientific studies on camera-motion, it is still an open question how to generalize this rule-of-thumb to digital still cameras as well as camera phones. In this paper, we first propose a generalized rule-of-thumb with the original rule-of-thumb as a special case when camera-motion can be approximated by a linear motion at 1.667 °/sec. We then use a gyroscope-based system to measure camera-motion patterns for two camera phones (one held with one hand and the other held in two hands) and one digital still camera. The results show that effective camera-motion function can be approximated very well by a linear function for exposure durations less than 100ms. While the effective camera-motion speed for camera phones (5.95 °/sec and 4.39 °/sec respectively) is significantly higher than that of digital still cameras (2.18 °/sec), it was found that holding a camera phone with two hands while taking pictures does reduce the amount of camera motion. It was also found that camera-motion not only varies significantly across subjects but also across captures for the same subject. Since camera phones have significantly higher motion and longer exposure durations than 35mm format film cameras and most digital still cameras, it is expected that many of the pictures taken by camera phones today will not meet the sharpness criteria used in 35mm film print. The mobile imaging industry is aggressively pursuing a smaller and smaller pixel size in order to meet the digital still cameras performance in terms of total pixels while retaining the small size needed for the mobile industry. This makes it increasingly more important to address the camera-motion challenge associated with smaller pixel size.

Evaluation of LED flash performance for camera phones

In this work, LED based flash solutions are evaluated for use in a camera phone application. The performance of a given flash solution is measured in terms of color accuracy and signal to noise ratio (SNR), both of which are standard test methods used in industry. Early in a camera phone design before completed camera modules are available, color accuracy and SNR are evaluated through a model which is based on knowledge of a given image sensors color response as well as the power spectral distribution of the flash. Later in the design, when working camera modules become available, the evaluation is performed through direct measurement. These direct measurements are also used to verify the results of the aforementioned model. Color accuracy, SNR, how they are related, and the compromise between the two are discussed as well as the efficiency at which electrical power is converted to light that is detectable by the image sensor. Though many issues remain to be investigated, measuring color accuracy and SNR provides an evaluation method that builds on developed techniques and provides a practical foundation for flash evaluation as it applies to the camera phone industry.