Ramsin Khoshabeh

An Augmented Lagrangian Method for Total Variation Video Restoration

This paper presents a fast algorithm for restoring video sequences. The proposed algorithm, as opposed to existing methods, does not consider video restoration as a sequence of image restoration problems. Rather, it treats a video sequence as a space-time volume and poses a space-time total variation regularization to enhance the smoothness of the solution. The optimization problem is solved by transforming the original unconstrained minimization problem to an equivalent constrained minimization problem. An augmented Lagrangian method is used to handle the constraints, and an alternating direction method is used to iteratively find solutions to the subproblems. The proposed algorithm has a wide range of applications, including video deblurring and denoising, video disparity refinement, and hot-air turbulence effect reduction.

SLAP widgets: bridging the gap between virtual and physical controls on tabletops

We present Silicone iLluminated Active Peripherals (SLAP), a system of tangible, translucent widgets for use on multitouch tabletops. SLAP Widgets are cast from silicone or made of acrylic, and include sliders, knobs, keyboards, and buttons. They add tactile feedback to multi-touch tables, improving input accuracy. Using rear projection, SLAP Widgets can be relabeled dynamically, providing inexpensive, battery-free, and untethered augmentations. Furthermore, SLAP combines the flexibility of virtual objects with physical affordances. We evaluate how SLAP Widgets influence the user experience on tabletops compared to virtual controls. Empirical studies show that SLAPWidgets are easy to use and outperform virtual controls significantly in terms of accuracy and overall interaction time.

Spatio-temporal consistency in video disparity estimation

We present a novel stereo video disparity estimation method. The proposed method is a two-stage algorithm. During the first stage, initial disparity maps are computed in a frame-by-frame basis. In the second stage, the initial estimates are treated as a space-time volume. By setting up an l1-normed minimization problem with a novel three-dimensional total variation regularization, spatial smoothness and temporal consistency are handled simultaneously. Due to our unique formulation, any existing image disparity estimation technique may utilize our method as a post-processing step to refine noisy estimates or to be extended to videos. The proposed method shows superior speed, accuracy, and consistency compared to state-of-the-art algorithms.

Efficient stereo-to-multiview synthesis

From a rectified stereo image pair, the task of view synthesis is to generate images from any viewpoint along the baseline. The main difficulty of the problem is how to fill occluded regions. In this paper, we present a new method for view synthesis that is both fast and accurate. Occlusions are filled using color and disparity information to produce consistent pixel estimates. Results are comparable to current state-of-the-art methods in terms of objective measures while computation time is drastically reduced. This work has applications in free-viewpoint television, angular scalability for 3D video coding/decoding, and stereo-to-multiview conversion.

Multi-camera Based Traffic Flow Characterization & Classification

We describe a system that employs the use of an omnidirectional camera in tandem with a pan-tilt-zoom (PTZ) camera in order to characterize traffic flows, analyze vehicles, and detect and capture anomalous behaviors. The framework is such that we can generate long-term statistics of traffic patterns while still monitoring unusual activity, even apart from the traffic zone. We use the PTZ in conjunction with the omni camera in order to perform classification analysis at coarse and fine levels. The omni performs the coarse classification and using information from that camera, the PTZ is able to perform refined classifications while actively moving throughout the scene.

An augmented Lagrangian method for video restoration

This paper presents a fast algorithm for restoring video sequences. The proposed algorithm, as opposed to existing methods, does not consider video restoration as a sequence of image restoration problems. Rather, it treats a video sequence as a space-time volume and poses a space-time total variation regularization to enhance the smoothness of the solution. The optimization problem is solved by transforming the original unconstrained minimization problem to an equivalent constrained minimization problem. An augmented Lagrangian method is used to handle the constraints, and an alternating direction method (ADM) is used to iteratively find solutions of the subproblems. The proposed algorithm has a wide range of applications, including video deblurring and denoising, disparity map refinement, and reducing hot-air turbulence effects.

Multiview Glasses-Free 3-D Laparoscopy

Numerous attempts have been made to introduce three-dimensional (3-D) video systems into clinical routine, particularly for surgeries. The drawback with all of them thus far has been the fact that they require users to wear cumbersome glasses in order to receive the advantage of stereoscopy. In this study, we present, to our best knowledge, the worlds very first laparoscopic surgical system that delivers glasses-free multiview 3-D in high-definition (HD) resolution. In addition to quantitative evaluations of the individual video processing components in our previous studies, we perform an initial subjective study with laparoscopically experienced surgeons, yielding very promising results. This study is still preliminary and requires further evaluation before the system may be introduced into standard surgical routine. Yet, we have developed a fully functioning prototype and successfully demonstrated its potential to numerous surgeons.

Spatially consistent view synthesis with coordinate alignment

In this paper, we propose a novel method that uses coordinate alignment and background pixel extraction to synthesize highly accurate and spatially consistent intermediate views from a pair of stereo images and disparity maps. In contrast to the traditional depth image-based rendering (DIBR) method, where useful background pixels are discarded in the warping process, the proposed method extracts these background pixels and uses them as candidates for an exemplar-based image in-painting technique (EBIIT) to synthesize realistic content in disocclusion regions. Our second contribution is a coordinate alignment algorithm that aligns disocclusion regions in each view together and simultaneously synthesizes disocclusion regions to enhance spatial consistency across all virtual views. The proposed method compares favorably in quantitative measures to those obtained by existing techniques and has superior potential for stereo-to-multiview conversion.

Spatio-temporal interest points for video analysis

In this paper, we discuss the need for an effective representation of video data to aid analysis of large datasets of video clips and describe a prototype developed to explore the use of spatio-temporal interest points for action recognition. Our focus is on ways that computation can assist analysis.