Scott Sorensen

Improving calibration of thermal stereo cameras using heated calibration board

Calibration of stereo cameras is important for accurate 3D reconstruction. For standard color cameras there are many available tools and algorithms for accurate calibration, such as detecting corners of chessboard patterns on planar calibration boards. When viewed in thermal imagery, these chessboard patterns are difficult to detect due to uniform temperature between the white and black squares. Previous techniques involve creating a custom calibration board using multiple materials. We propose improvements to a method that does not require a custom calibration board. Our method is made more reliable by using an iterative pre-processing technique to enhance contrast and a ceramic tile backing to retain heat longer. We present results which show our calibration board retains heat to reliably detect corners for over 10 minutes; our method performs well in real calibration trials.

Reconstruction of textureless regions using structure from motion and image-based interpolation

Techniques based on the well-studied approaches of structure from motion and bundle adjustment are very robust for scenes with texture. In scenes with little texture information these approaches can fail. Shape from shading determines the shape of an object up to a scale from a single image, and performs better than structure from motion methods in textureless regions. We propose using Gradient Constrained Interpolation to estimate a dense point cloud where holes are caused by regions of low texture during structure from motion reconstruction. Our technique is demonstrated to show good results in both synthetic and real data and outperforms methods which do not use image information.

Material classification with thermal imagery

Material classification is an important area of research in computer vision. Typical algorithms use color and texture information for classification, but there are problems due to varying lighting conditions and diversity of colors in a single material class. In this work we study the use of long wave infrared (i.e. thermal) imagery for material classification. Thermal imagery has the benefit of relative invariance to color changes, invariance to lighting conditions, and can even work in the dark. We collect a database of 21 different material classes with both color and thermal imagery. We develop a set of features that describe water permeation and heating/cooling properties, and test several variations on these methods to obtain our final classifier. The results show that the proposed method outperforms typical color and texture features, and when combined with color information, the results are improved further.

Shape from stereo and shading by gradient constrained interpolation

Textureless regions, though error prone in stereo, may contain shading information that may be exploited. Shape from shading (SFS) results relate to world coordinates by arbitrary scaling factors which are difficult to estimate. We propose a method for estimating dense disparities from sparse correspondences using SFS cues. We show that SFS can impose constraints on the gradient of disparity in textureless regions with constant albedo. Gradient Constrained Interpolation (GCI), which solves the estimation problem in one dimension, is presented. We efficiently generate paths between correspondences that cover the image and then use GCI to fill the pixels in between. Results are presented on real and synthetic images, and provide quantitative evaluations to show that the method outperforms baseline methods.

Iterative Reconstruction of Large Scenes Using Heterogeneous Feature Tracking

With image capturing technology growing ubiquitous in consumer products and scientific studies, there is a corresponding growth in the applications that utilize scene structure for deriving information. This trend has also been reflected in the plethora of recent studies on reconstruction using robust structure from motion, bundle adjustment, and related techniques. Most of these studies, however, have concentrated on unstructured collections of images. In this paper, we propose a feature tracking and reconstruction framework for structured image collections using heterogenous features. This is motivated by the observation that images contain a small number of features that are fast/easy to track and a large number of features that are difficult/slow to track. By tracking these separately, we show that we can not only improve the tracking speed, but also improve the tracking accuracy by using a camera geometry based descriptor. We demonstrate this on a new challenging dataset which contains images of Arctic sea ice. The reconstruction pipeline constructed using the proposed method provides near real time reconstruction of the scene, enabling the user to parse vast amounts of data rapidly. Quantitative comparisons with baseline SFM techniques show that reconstruction accuracy does not suffer.

Refractive stereo ray tracing for reconstructing underwater structures

Underwater objects behind a refractive surface pose problems for traditional 3D reconstruction techniques. Scenes where underwater objects are visible from the surface are commonplace, however the refraction of light causes 3D points in these scenes to project non-linearly. Refractive Stereo Ray Tracing allows for accurate reconstruction by modeling the refraction of light. Our approach uses techniques from ray tracing to compute the 3D position of points behind a refractive surface. This technique aims to reconstruct underwater structures in situations where access to the water is dangerous or cost prohibitive. Experimental results in real and synthetic scenes show this technique effectively handles refraction.

Multimodal Stereo Vision For Reconstruction In The Presence Of Reflection.

Reflective and specular surfaces are problematic for traditional reconstruction techniques. Light projects non-linearly in scenes with these surfaces, and existing techniques to model this are poorly suited for real world applications. Accurately modeling the reflective surface is difficult without complete knowledge of the scene. To overcome this problem, we propose using different modalities of stereo vision to capture both the reflecting surface and the reflected scene. Using a four camera system consisting of a pair of visible wavelength cameras and a pair of long wave infrared cameras, we accurately reconstruct the reflective surface and ray trace reflected correspondences in the complementary modality. This approach allows for 3D reconstruction in the presence of a reflection, and does not require complete knowledge of the scene.

Dog breed classification via landmarks

Object recognition is an important problem with a wide range of applications. It is also a challenging problem, especially for animal categorization as the differences among breeds can be subtle. In this paper, based on statistical techniques for landmark-based shape representation, we propose to model the shape of dog breed as points on the Grassmann manifold. We consider the dog breed categorization as the classification problem on this manifold. The proposed scheme is tested on a dataset including 8,351 images of 133 different breeds. Experimental results demonstrate the advocated scheme outperforms state of the art approaches by nearly 20%.

A Virtual Reality Framework for Multimodal Imagery for Vessels in Polar Regions

Maintaining total awareness when maneuvering an ice-breaking vessel is key to its safe operation. Camera systems are commonly used to augment the capabilities of those piloting the vessel, but rarely are these camera systems used beyond simple video feeds. To aid in visualization for decision making and operation, we present a scheme for combining multiple modalities of imagery into a cohesive Virtual Reality application which provides the user with an immersive, real scale, view of conditions around a research vessel operating in polar waters. The system incorporates imagery from a \(360^{\circ }\) Long-wave Infrared camera as well as an optical band stereo camera system. The Virtual Reality application allows the operator multiple natural ways of interacting with and observing the data, as well as provides a framework for further inputs and derived observations.

Large-Scale Tracking for Images With Few Textures

Image tracking provides crucial insight for the image motion, which generates essential information for incremental structure-from-motion reconstruction and camera pose estimation. Typical usages, such as 3D reconstruction and visual odometry, all rely on robust and accurate local feature tracking through consecutive images. Current algorithms realize feature tracking through matching features extracted from discriminant textures in the images, for which distinctive image content is required to obtain accurate feature matching. For images with few textures, usually, an insufficient number of features are extracted to perform reliable tracking in a series of sequential images. We propose a method that makes use of a limited number of discriminate features to explore other features without strong discriminant power. We develop a feature integrating surrounding salient points distribution knowledge, raw pixel value, and coordinate information to discover a significant amount of features in weakly textured areas in an image. We also incorporate epipolar geometry in the feature correspondence calculation by taking the distance from the matching candidate to its corresponding points epipolar line into account. To reduce the number of unreliable features, we project the estimated 3D points back to the images. The reprojection error is standardized according to the 3D points depth, which reduces the bias introduced by the object distance to the camera. We conduct experiments on a large dataset of Arctic sea ice images, mainly composed by planes of ices and sea water. The experimental results demonstrate that our method can perform fast and accurate tracking in weakly textured images.