Aziz Umit Batur

Eigenface-domain super-resolution for face recognition

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing step to obtain a high-resolution image that is later passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose to transfer the super-resolution reconstruction from pixel domain to a lower dimensional face space. Such an approach has the advantage of a significant decrease in the computational complexity of the super-resolution reconstruction. The reconstruction algorithm no longer tries to obtain a visually improved high-quality image, but instead constructs the information required by the recognition system directly in the low dimensional domain without any unnecessary overhead. In addition, we show that face-space super-resolution is more robust to registration errors and noise than pixel-domain super-resolution because of the addition of model-based constraints.

Adaptive active appearance models

The active appearance model (AAM) is a powerful tool for modeling images of deformable objects and has been successfully used in a variety of alignment, tracking, and recognition applications. AAM uses subspace-based deformable models to represent the images of a certain object class. In general, fitting such complicated models to previously unseen images using standard optimization techniques is a computationally complex task because the gradient matrix has to be numerically computed at every iteration. The critical feature of AAM is a fast convergence scheme which assumes that the gradient matrix is fixed around the optimal coefficients for all images. Our work in this paper starts with the observation that such a fixed gradient matrix inevitably specializes to a certain region in the texture space, and the fixed gradient matrix is not a good estimate of the actual gradient as the target texture moves away from this region. Hence, we propose an adaptive AAM algorithm that linearly adapts the gradient matrix according to the composition of the target images texture to obtain a better estimate for the actual gradient. We show that the adaptive AAM significantly outperforms the basic AAM, especially in images that are particularly challenging for the basic algorithm. In terms of speed and accuracy, the idea of a linearly adaptive gradient matrix presented in this paper provides an interesting compromise between a standard optimization technique that recomputes the gradient at every iteration and the fixed gradient matrix approach of the basic AAM.

Linear subspaces for illumination robust face recognition

In this paper, we present a segmented linear subspace model for face recognition that is robust under varying illumination conditions. The algorithm generalizes the 3D illumination subspace model by segmenting the image into regions that have surface normals whose directions are close to each other. This segmentation is performed using a K-means clustering algorithm and requires only a few training images under different illuminations. When the linear subspace model is applied to the segmented image, recognition is robust to attached and cast shadows, and the recognition rate is equal to that of computationally more complex systems that require constructing the 3D surface of the face.

Eigenface-based super-resolution for face recognition

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed that attempt to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing system to obtain a high resolution image that can later be passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose embedding the super-resolution algorithm into the face recognition system so that super-resolution is not performed in the pixel domain, but is instead performed in a reduced dimensional domain. The advantage of such an approach is a significant decrease in the computational complexity of the super-resolution algorithm because the algorithm no longer tries to construct a visually improved high quality image, but instead constructs the information required by the recognition algorithm directly in the lower dimensional domain without any unnecessary overhead.

A Surround View Camera Solution for Embedded Systems

Automotive surround view camera system is an emerging automotive ADAS (Advanced Driver Assistance System) technology that assists the driver in parking the vehicle safely by allowing him/her to see a top-down view of the 360 degree surroundings of the vehicle. Such a system normally consists of four to six wide-angle (fish-eye lens) cameras mounted around the vehicle, each facing a different direction. From these camera inputs, a composite bird-eye view of the vehicle is synthesized and shown to the driver in real-time during parking. In this paper, we present a surround view camera solution that consists of three key algorithm components: geometric alignment, photometric alignment, and composite view synthesis. Our solution produces a seamlessly stitched bird-eye view of the vehicle from four cameras. It runs real-time on DSP C66x producing an 880x1080 output video at 30 fps.

Video stabilization and rolling shutter distortion reduction

This paper presents an algorithm that stabilizes video and reduces rolling shutter distortions using a six-parameter affine model that explicitly contains parameters for translation, rotation, scaling, and skew to describe transformations between frames. Rolling shutter distortions, including wobble, skew and vertical scaling distortions, together with both translational and rotational jitter are corrected by estimating the parameters of the model and performing compensating transformations based on those estimates. The results show the benefits of the proposed algorithm quantified by the Interframe Transformation Fidelity (ITF) metric.

Video Stabilization with Optimized Motion Estimation Resolution

We propose a video stabilization algorithm that removes camera jitter in complex scenes with moving objects. Our algorithm is based on analyzing the content of the video according to the accumulated motion between the scene and the camera, which provides a simple and robust way of estimating jitter in the presence of camera panning and moving objects. We also describe a method of reducing the computational complexity of our algorithm by using a hierarchical clustering technique that optimizes the motion estimation resolution at different scene locations.

Segmented Linear Subspaces for Illumination-Robust Face Recognition

All images of a convex Lambertian surface captured with a fixed pose under varying illumination are known to lie in a convex cone in the image space that is called the illumination cone. Since this cone model is too complex to be built in practice, researchers have attempted to approximate it with simpler models. In this paper, we propose a segmented linear subspace model to approximate the cone. Our idea of segmentation is based on the fact that the success of low dimensional linear subspace approximations of the illumination cone increases if the directions of the surface normals get close to each other. Hence, we propose to cluster the image pixels according to their surface normal directions and to approximate the cone with a linear subspace for each of these clusters separately. We perform statistical performance evaluation experiments to compare our system to other popular systems and demonstrate that the performance increase we obtain is statistically significant.

A real-time auto white balance algorithm for mobile phone cameras

Auto white balance (AWB), a critical component of the image pipeline in digital cameras, is responsible for producing accurate color by automatically removing the undersired color cast introduced by the illumination. AWB estimates the scene illumination by processing the digital values of the pixels in the captured image; therefore, objects colors in the scene are often confused with the color of the light source, leading to wrong color cast in white balanced images. In this paper, we propose a color histogram based AWB algorithm that is capable of producing accurate color in the presence of dominant object colors. Our approach is based on a statistical estimation of the probabilities of colors in natural scenes under different illuminations. These statistics can be easily collected during the AWB calibration phase using standard equipment. The proposed AWB algorithm is computationally efficient and runs real-time in a mobile phone camera.

Joint focus stacking and high dynamic range imaging

Focus stacking and high dynamic range (HDR) imaging are two paradigms of computational photography. Focus stacking aims to produce an image with greater depth of field (DOF) from a set of images taken with different focus distances, whereas HDR imaging aims to produce an image with higher dynamic range from a set of images taken with different exposure settings. In this paper, we present an algorithm which combines focus stacking and HDR imaging in order to produce an image with both higher dynamic range and greater DOF than any of the input images. The proposed algorithm includes two main parts: (i) joint photometric and geometric registration and (ii) joint focus stacking and HDR image creation. In the first part, images are first photometrically registered using an algorithm that is insensitive to small geometric variations, and then geometrically registered using an optical flow algorithm. In the second part, images are merged through weighted averaging, where the weights depend on both local sharpness and exposure information. We provide experimental results with real data to illustrate the algorithm. The algorithm is also implemented on a smartphone with Android operating system.