Thommen Korah

Analysis of Building Textures for Reconstructing Partially Occluded Facades

As part of an architectural modeling project, this paper investigates the problem of understanding and manipulating images of buildings. Our primary motivation is to automatically detect and seamlessly remove unwanted foreground elements from urban scenes. Without explicit handling, these objects will appear pasted as artifacts on the model. Recovering the building facade in a video sequence is relatively simple because parallax induces foreground/background depth layers, but here we consider static images only. We develop a series of methods that enable foreground removal from images of buildings or brick walls. The key insight is to use a prioriknowledge about grid patterns on building facades that can be modeled as Near Regular Textures (NRT). We describe a Markov Random Field (MRF) model for such textures and introduce a Markov Chain Monte Carlo (MCMC) optimization procedure for discovering them. This simple spatial rule is then used as a starting point for inference of missing windows, facade segmentation, outlier identification, and foreground removal.

On-Vehicle and Aerial Texture Analysis for Vision-Based Desert Road Following

We present two components of a vision-based approach to autonomous driving on and near rural and desert roads. The first component comprises fast processing of live vehicle camera video to robustly extract linear direction and midline estimates of marginal roads. The second uses satellite imagery immediately surrounding the vehicle’s GPS position to trace the road ahead for curve and corner anticipation, and to inform the vehicle planner of a nearby road when traveling cross-conalysis: on-board, they are employed to ?nd ruts and trauntry. The algorithms are built upon Gabor wavelet filters for texture acks from which the road vanishing point can be inferred via Houghstyle voting, and aerially, they localize the edges of low-contrast road contours. Mechanisms for both modules to determine whether the vehicle is currently on- or off-road are also explained. Our system’s efficacy is illustrated for several difficult datasets, including a log from one vehicle’s run during the 2004 DARPA Grand Challenge.

Spatiotemporal Inpainting for Recovering Texture Maps of Occluded Building Facades

We present a technique for constructing a ldquocleanrdquo texture map of a partially occluded building facade from a series of images taken from a moving camera. Building regions blocked by trees, signs, people, and other foreground objects in a minority of views can be recovered via temporal median filtering on a registered image mosaic of the planar facade. However, when such areas are occluded in the majority of camera views, appearance information from other visible portions of the facade provides a critical cue to correctly complete the mosaic. In this paper, we apply a robust measure of spread to infer whether a particular mosaic pixel is occluded in a majority of views, and introduce a novel spatiotemporal timeline-based inpainting algorithm that uses appearance and motion cues in order to fill the texture map in majority-occluded regions. We describe methods for automatically training appearance-based classifiers from a coarse motion-based segmentation to efficiently recognize foreground and background patches in static imagery. Results of recovered building facades are shown for various sequences.

2D Lattice Extraction from Structured Environments

In this paper we investigate the problem of automatically detecting 2D grid structures such as windows on building facades from images taken in urban settings. The key assumption that the background is strongly structured allows searching for near-regular textures in the image. We describe a probabilistic framework using Markov Random Field modeling and Markov Chain Monte Carlo (MCMC) optimization to explicitly recognize and group rectangular structures that appear in a grid-like pattern. Results on a variety of images of building facades are shown.

Probabilistic contour extraction with model-switching for vehicle localization

Over the past few years, global positioning systems (GPS) have been increasingly used in passenger and commercial vehicles for navigation and vehicle tracking purposes. In practice, GPS systems are prone to systematic errors and intermittent drop-outs that degrade the accuracy of the sensor. In this work, we describe an approach to localizing vehicles with respect to the road given erroneous sensor measurements using only aerial images. Our method works on both urban and rural areas, while being robust to a number of occlusions and shadows. The spatial tracker incorporates multiple measurement models with varying constraints, automatically detecting and switching to the appropriate model. We demonstrate our technique by correcting in real-time highly inaccurate GPS readings collected while driving in diverse areas.

Aligning sequences from multiple cameras

This paper studies the problem of aligning images from multiple cameras with minimally- or non-overlapping fields of view using frame-to-frame transformations calculated for sequences from each camera. We examine implementation issues for the algorithm of Caspi and Irani that performs the alignment for two cameras which are fixed relative to each other and have approximately the same center of projection. Furthermore, we extend it to compute the camera-to-camera homographies for an N/spl ges/2 multi-camera network by simultaneously solving for all parameters of the unknown transformations. This enforces tighter constraints on the solution than performing the alignment for each pair independently. We show the efficacy of the approach on both synthetic as well as real sequences captured using a polycamera built in our lab. The aligned images can be mosaiced together to obtain a wider field of view virtual camera for subsequent processing.

PCA-Based recognition for efficient inpainting

We present a technique for efficiently constructing a “clean” texture map of a partially occluded building facade from a series of images taken by a moving camera. After a robust registration procedure, building regions blocked by trees, signs, people, and other foreground objects are automatically inferred via the median absolute deviation of colors from different source images mapping to the same mosaic pixels. In previous work we extended an existing non-parametric inpainting algorithm for filling such holes to incorporate spatiotemporal appearance and motion cues in order to correctly replace the outlier pixels of the texture map. In contrast to other inpainting techniques that perform an exhaustive search over the image, in this work we introduce a principal components-based method that learns to recognize patches that locally adhere to the properties of the building being mapped, resulting in a significant performance boost with results of indistinguishable quality. Results are demonstrated on sequences where previous stitching and inpainting algorithms fail.

Improving spatiotemporal inpainting with layer appearance models

The problem of removing blemishes in mosaics of building facades caused by foreground objects such as trees may be framed in terms of inpainting. Affected regions are first automatically segmented and then inpainted away using a combination of cues from unoccluded, temporally adjacent views of the same building patch, as well as surrounding unoccluded patches in the same frame. Discriminating the building layer from those containing foreground features is most directly accomplished through parallax due to camera motion over the sequence. However, the intricacy of tree silhouettes often complicates accurate motion-based segmentation, especially along their narrower branches. In this work we describe methods for automatically training appearance-based classifiers from a coarse motion-based segmentation to recognize foreground patches in static imagery and thereby improve the quality of the final mosaic. A local technique for photometric adjustment of inpainted patches which compensates for exposure variations between frames is also discussed.