Markku Vehvilainen

Accelerating image recognition on mobile devices using GPGPU

The future multi-modal user interfaces of battery-powered mobile devices are expected to require computationally costly image analysis techniques. The use of Graphic Processing Units for computing is very well suited for parallel processing and the addition of programmable stages and high precision arithmetic provide for opportunities to implement energy-efficient complete algorithms. At the moment the first mobile graphics accelerators with programmable pipelines are available, enabling the GPGPU implementation of several image processing algorithms. In this context, we consider a face tracking approach that uses efficient gray-scale invariant texture features and boosting. The solution is based on the Local Binary Pattern (LBP) features and makes use of the GPU on the pre-processing and feature extraction phase. We have implemented a series of image processing techniques in the shader language of OpenGL ES 2.0, compiled them for a mobile graphics processing unit and performed tests on a mobile application processor platform (OMAP3530). In our contribution, we describe the challenges of designing on a mobile platform, present the performance achieved and provide measurement results for the actual power consumption in comparison to using the CPU (ARM) on the same platform.

Motion Blur Identification Based on Differently Exposed Images

In this paper we introduce a new method of motion blur identification that relies on the availability of two, differently exposed, image shots of the same scene. The proposed approach exploits the difference in the degradation models of the two images in order to identify the point spread function (PSF) corresponding to the motion blur, that may affect the longer exposed image shot. The algorithm is demonstrated through a series of experiments that reveal its ability to identify the motion blur PSF even in the presence of heavy degradations of the two observed images.

Image Stabilization Based on Fusing the Visual Information in Differently Exposed Images

The objective of image stabilization is to prevent or remove the motion blur degradation from images. We introduce a new approach to image stabilization based on combining information available in two differently exposed images of the same scene. In addition to the image normally captured by the system, with an exposure time determined by the illumination conditions, a very shortly exposed image is also acquired. The difference between the exposure times of the two images determines differences in their degradations which are exploited in order to recover the original image of the scene. We formulate the problem as a maximum a posteriori (MAP) estimation based on the degradation models of the two observed images, as well as by imposing an edge-preserving image prior. The proposed method is demonstrated through a series of simulation experiments, and visual examples on natural images.

High dynamic range imaging on mobile devices

Natural scenes usually produce radiance maps that have a dynamic range much larger than the dynamic range of the imaging sensors. Due to this fact the captured images, almost always, contain under-exposed and saturated regions. Among several solutions, proposed in the open literature, the multi-frame approaches have been shown to produce high quality results by combining several shots of the same scene, captured at different exposure times. Here we introduce a low complexity multi-frame approach suitable for mobile implementations. We have implemented our method in Symbian OS in a Nokia cameraphone and the results obtained with our proposed system are shown in the paper.

Method of Motion Estimation for Image Stabilization

In this paper we introduce a novel approach to global motion estimation for image stabilization application. The method is robust to image degradations characteristic to image stabilization, e.g. image blur caused by motion or out of focus. In addition, due to its low computational complexity, the proposed method could be included in a real-time digital image stabilization system. The ability of the proposed registration approach to capture the global motion of the camera in the presence of image degradations and outliers, have been evaluated through a large number of experiments. The results reveal that, in spite of its lower computational complexity, the proposed method achieves sub-pixel motion estimation accuracy, close to the performance achieved by the state of the art approaches to image registration

Fast nonlocal means for image denoising

Due to camera module miniaturization, the pixel area of the digital sensors decreases which decreases also the signal to noise ratio in the captured images. As a consequence, image de-noising is still an important topic in digital image processing field. In this paper we address the problem of image de-noising using the nonlocal means algorithm. This method has excellent de-noising properties but at the expense of increasing the computational complexity. We propose here a novel approach that provides similar filtering capabilities with much less computational effort and shorter processing time. Our proposed algorithm is compared with the nonlocal means algorithm and with another fast implementation, recently reported, in terms of processing time and noise reduction capability (from both visual impression and mean squared error points of view). The comparative results are presented for artificially degraded images and also for images obtained with a camera phone.

Cross-color BM3D filtering of noisy raw data

Color image reconstruction from noisy color filter array (CFA) data is considered. A modification of the Block Matching 3D (BM3D) [2] filter for CFA data denoising utilizing cross-color correlations is proposed. Denoised images are then demosaicked by algorithms developed for noise-free data leading to state-of-the-art performance for both Gaussian and Poissonian noise models.

Graphics hardware accelerated panorama builder for mobile phones

Modern mobile communication devices frequently contain built-in cameras allowing users to capture highresolution still images, but at the same time the imaging applications are facing both usability and throughput bottlenecks. The difficulties in taking ad hoc pictures of printed paper documents with multi-megapixel cellular phone cameras on a common business use case, illustrate these problems for anyone. The result can be examined only after several seconds, and is often blurry, so a new picture is needed, although the view-finder image had looked good. The process can be a frustrating one with waits and the user not being able to predict the quality beforehand. The problems can be traced to the processor speed and camera resolution mismatch, and application interactivity demands. In this context we analyze building mosaic images of printed documents from frames selected from VGA resolution (640x480 pixel) video. High interactivity is achieved by providing real-time feedback on the quality, while simultaneously guiding the user actions. The graphics processing unit of the mobile device can be used to speed up the reconstruction computations. To demonstrate the viability of the concept, we present an interactive document scanning application implemented on a Nokia N95 mobile phone.

Multichannel image deblurring of raw color components

This paper presents a novel multichannel image restoration algorithm. The main idea is to develop practical approaches to reduce optical blur from noisy observations produced by the sensor of a camera phone. An iterative deconvolution is applied separately to each color channel directly on the raw data. We use a modified iterative Landweber algorithm combined with an adaptive denoising technique. The adaptive denoising is based on local polynomial approximation (LPA) operating on data windows selected by the rule of intersection of confidence intervals (ICI). In order to avoid false coloring due to independent component filtering in the RGB space, we have integrated a novel saturation control mechanism that smoothly attenuates the high-pass filtering near saturated regions. It is shown by simulations that the proposed filtering is robust with respect to errors in point-spread function and approximated noise models. Experimental results show that the proposed processing technique produces significant improvement in perceived image resolution.

Constrained Unsharp Masking for Image Enhancement

In this paper we present a cost-effective solution for combined de-noising and sharpening of digital images. Our method combines the unsharp masking and sigma filtering techniques through a regularization mechanism thus ensuring effective noise reduction and edge enhancement in the processed image. We describe our method in detail and we analyze the proposed implementation through extensive experiments done in various scenarios. Due to its low computational complexity the proposed method is well suited for mobile implementations.