Alejandro Troccoli

Robust stereo with flash and no-flash image pairs

We propose a new stereo technique using a pair of flash and no-flash stereo images that is both efficient and robust in handling occlusion boundaries. Our work is motivated by the observation that the brightness variations introduced by the flash can provide a robust cue for establishing stereo matches at occlusion boundaries. This photometric cue is computed per pixel, and though on its own is not robust to reliably resolve depth, it can provide a new discriminant to support patch-based stereo matching algorithms. Our experiments using a hand-held Fujifilm W3 3D camera show satisfying stereo performance over a variety of scenes, including several outdoor scenes.

MLMD: Maximum Likelihood Mixture Decoupling for Fast and Accurate Point Cloud Registration

Registration of Point Cloud Data (PCD) forms a core component of many 3D vision algorithms such as object matching and environment reconstruction. In this paper, we introduce a PCD registration algorithm that utilizes Gaussian Mixture Models (GMM) and a novel dual-mode parameter optimization technique which we call mixture decoupling. We show how this decoupling technique facilitates both faster and more robust registration by first optimizing over the mixture parameters (decoupling the mixture weights, means, and co variances from the points) before optimizing over the 6 DOF registration parameters. Furthermore, we frame both the decoupling and registration process inside a unified, dual-mode Expectation Maximization (EM) framework, for which we derive a Maximum Likelihood Estimation (MLE) solution along with a parallel implementation on the GPU. We evaluate our MLE-based mixture decoupling (MLMD) registration method over both synthetic and real data, showing better convergence for a wider range of initial conditions and higher speeds than previous state of the art methods.

A Non-Linear Filter for Gyroscope-Based Video Stabilization

We present a method for video stabilization and rolling-shutter correction for videos captured on mobile devices. The method uses the data from an on-board gyroscope to track the camera’s angular velocity, and can run in real time within the camera capture pipeline. We remove small motions and rolling-shutter distortions due to hand shake, creating the impression of a video shot on a tripod. For larger motions, we filter the camera’s angular velocity to produce a smooth output. To meet the latency constraints of a real-time camera capture pipeline, our filter operates on a small temporal window of three to five frames. Our algorithm performs better than the previous work that uses a gyroscope to stabilize a video stream, and at a similar level with respect to current feature-based methods.

Locally non-rigid registration for mobile HDR photography

Image registration for stack-based HDR photography is challenging. If not properly accounted for, camera motion and scene changes result in artifacts in the composite image. Unfortunately, existing methods to address this problem are either accurate, but too slow for mobile devices, or fast, but prone to failing. We propose a method that fills this void: our approach is extremely fast - under 700ms on a commercial tablet for a pair of 5MP images - and prevents the artifacts that arise from insufficient registration quality.

FCam for multiple cameras

The Frankencamera (FCam) architecture and API enables precise control over the camera in computational photography applications. We present an extension to FCam API for systems equipped with multiple cameras. The proposed extension allows for an enumeration of cameras and their corresponding properties, such as position or orientation. In addition, we explicitly support camera synchronization, either through hardware mechanisms or software primitives. If hardware synchronization is available, cameras can be grouped together under a concept of a multi-sensor. Otherwise, multiple camera streams are scheduled asynchronously and synchronized using our software control primitives.

Mobile Computational Photography with FCam

In this chapter we cover the FCam (short for Frankencamera) architecture and API for computational cameras.We begin with the motivation, which is flexible programming of cameras, especially of camera phones and tablets. We cover the API and several example programs that run on the NVIDIA Tegra 3 prototype tablet and the Nokia N900 and N9 Linux-based phones. We discuss the implementation and porting of FCam to different platforms. We also describe how FCam has been used at many universities to teach computational photography.

FCam: an architecture and API for computational cameras

This course is designed to demonstrate the use of the FCam, an Application Programming Interface (API) which allows for easy and precise control of camera systems. It lets programmers implement advanced computational photography applications. Such applications even allow the modification of basic camera routines such as metering, which are hidden inside black boxes in most camera systems. FCam is also an excellent tool for teaching and research on interactive mobile computational photography. The API has been open-sourced, and is currently implemented for Nokia N900 and an experimental Frankencamera (the F2). The Frankencamera architecture for computational cameras was presented at SIGGRAPH 2010. By end-2011, the FCam will hopefully be available on Nokia#8482; Linux phones, NVIDIA#8482; Tegra 3 development boards, and F3, the next version of the Stanford experimental camera. The F3 will be distributed to research universities within the US along with courseware in a National Science Foundation (NSF)-sponsored project. NVIDIA Tegras and Nokia#8482; Linux phones are available globally.