Fay Huang

Panoramic Imaging - Sensor-Line Cameras and Laser Range-Finders -

Preface. Series Preface. Website and Exercises. List of Symbols. 1. Introduction. 1.1 Panoramas 1.2 Panoramic Paintings 1.3 Panoramic or Wide-Angle Photographs 1.4 Digital Panoramas 1.5 Striving for Accuracy 1.6 Exercises 1.7 Further Reading 2. Cameras and Sensors. 2.1 Camera Models 2.2 Optics 2.3 Sensor Models 2.4 Examples and Challenges 2.5 Exercises 2.6 Further Reading 3. Spatial Alignments. 3.1 Mathematical Fundamentals 3.2 Central Projection:World into Image Plane 3.3 Classification of Panoramas 3.4 Coordinate Systems for Panoramas 3.5 General Projection Formula for Cylindrical Panorama 3.6 Rotating Cameras 3.7 Mappings between Different Image Surfaces 3.8 Laser Range-Finder 3.9 Exercises 3.10 Further Reading 4. Epipolar Geometry. 4.1 General Epipolar Curve Equation 4.2 Constrained Poses of Cameras 4.3 Exercises 4.4 Further Reading 5. Sensor Calibration. 5.1 Basics 5.2 Preprocesses for a Rotating Sensor-Line Camera 5.3 A Least-Square Error Optimization Calibration Procedure 5.4 Geometric Dependencies of R and w 5.5 Error Components in LRF Data 5.6 Exercises 5.7 Further Reading 6. Spatial Sampling. 6.1 Stereo Panoramas 6.2 Sampling Structure 6.3 Spatial Resolution 6.4 Distances between Spatial Samples 6.5 Exercises 6.6 Further Reading 7. Image Quality Control. 7.1 Two Requirements 7.2 Terminology 7.3 Parameter Optimization 7.4 Error Analysis 7.5 Exercises 7.6 Further Reading 8. Sensor Analysis and Design. 8.1 Introduction 8.2 Scene Composition Analysis 8.3 Stereoacuity Analysis 8.4 Specification of Camera Parameters 8.5 Exercises 8.6 Further Reading 9. 3D Meshing and Visualization. 9.1 3D Graphics 9.2 Surface Modeling 9.3 More Techniques for Dealing with Digital Surfaces 9.4 Exercises 9.5 Further Reading 10. Data Fusion. 10.1 Determination of Camera Image Coordinates 10.2 Texture Mapping 10.3 High Resolution Orthophotos 10.4 Fusion of Panoramic Images and Airborne Data 10.5 Exercises 10.6 Further Reading References. Index.

Geometrical fundamentals of polycentric panoramas

This paper proposes polycentric panoramas as a general model of panoramic images. The model formalizes essential characteristics of panoramic geometry. It is able to describe a wide range of panoramic images, including those potentially of future interest, or previously introduced such as single-center, multi-perspective, or concentric panoramas. This paper presents geometrical fundamentals towards stereo applications based on sets of polycentric panoramas. We discuss the image acquisition model, epipolar geometry and a 3D reconstruction approach for this general model of polycentric panoramas. Our theorems on epipolar curve and 3D reconstruction hold for any pair of polycentric panoramas. Corollaries demonstrate that the proposed mathematical model clarifies the understanding and characterization of more specific models. Epipolar curves of special cases are illustrated on panoramic images acquired by a high resolution line-camera.

Moving foreground object detection via robust SIFT trajectories

In this paper, we present an automatic foreground object detection method for videos captured by freely moving cameras. While we focus on extracting a single foreground object of interest throughout a video sequence, our approach does not require any training data nor the interaction by the users. Based on the SIFT correspondence across video frames, we construct robust SIFT trajectories in terms of the calculated foreground feature point probability. Our foreground feature point probability is able to determine candidate foreground feature points in each frame, without the need of user interaction such as parameter or threshold tuning. Furthermore, we propose a probabilistic consensus foreground object template (CFOT), which is directly applied to the input video for moving object detection via template matching. Our CFOT can be used to detect the foreground object in videos captured by a fast moving camera, even if the contrast between the foreground and background regions is low. Moreover, our proposed method can be generalized to foreground object detection in dynamic backgrounds, and is robust to viewpoint changes across video frames. The contribution of this paper is trifold: (1) we provide a robust decision process to detect the foreground object of interest in videos with contrast and viewpoint variations; (2) our proposed method builds longer SIFT trajectories, and this is shown to be robust and effective for object detection tasks; and (3) the construction of our CFOT is not sensitive to the initial estimation of the foreground region of interest, while its use can achieve excellent foreground object detection results on real-world video data.

Epipolar Geometry in Polycentric Panoramas

This paper proposes a new and general model of panoramic images, namely polycentric panoramas, which formalizes the essential characteristics of panoramic image acquisition geometry. This new model is able to describe a wide range of panoramic images including those which have been previously introduced such as single-center, multi-perspective, or concentric panoramas [1, 5,11,14] and that are potentially of interest in further research. This paper presents a study of epipolar geometry for pairs of polycentric panoramas. The first and unique epipolar curve equation derived provides a unified approach for computing epipolar curves in more specific types of panoramic images. Examples of epipolar curves in different types of panoramic images are also discussed in the paper.

Stereo panorama acquisition and automatic image disparity adjustment for stereoscopic visualization

Image-based visualization is popular for various virtual tour applications, due to high-quality photorealism or simplicity for rendering. Stereo panorama representations of the virtual world are already a common part of this, either in small (computer screen) format or on large-scale stereo displays or screens. This paper discusses methods for determining optimum parameters, both for high-accuracy stereo panoramic image recording and displaying, with a special focus on automatic image disparity enhancement while displaying (e.g., including zooming) a stereo panorama. Experiments show that the discussed parameters are indeed critical for ensuring high-quality stereo viewing. Derived formulas in this study are applicable to various kinds of technologies for stereo panorama imaging or stereoscopic displaying.

Comparative Studies of Line-based Panoramic Camera Calibration

The calibration of a line-based panoramic camera can be split into two independent subtasks: first calibrate the effective focal length and the principal row, and second, calibrate the off-axis distance and the principal angle. The paper provides solutions for three different methods, and compares these methods based on experiments using a superhigh resolution line-based panoramic camera. It turns out that the second subtask is solved best if a straight-segment based approach is used, compared to point-based or correspondence-based calibration methods, all already known for traditional (planar) pinhole cameras, but not yet previously discussed for panoramic cameras.

The design of a stereo panorama camera for scenes of dynamic range

Existing stereo panorama cameras do not allow controllability of pictorial/scene composition and stereo acuity (depth levels) over dynamic 3D scene ranges. We specify the design of such a camera allowing this type of flexibility. Previous approaches to design panorama cameras even lack studies with respect to this important aspect, while other design issues such as epipolar geometry, optics optimization, or realization-oriented approximations have been investigated. Without incorporating the controllability into stereo panorama camera design, the poor quality of produced stereo panoramas is foreseeable (e.g. incoherence, cardboard-effect, dipopia etc.). The paper proposes a solution to incorporate controllability into previously discussed (Ishiguro et al., 1992; Wei ei al., 1999; Shum et al., 1999; Peleg et al., 2000) stereo panorama camera models. By using a stereo panorama camera equipped with the designed camera parameters according to our solution, the desired/expected pictorial composition and stereo acuity in resultant stereo panoramas can be ensured.

Monocular 3D Reconstruction of Objects Based on Cylindrical Panoramas

This paper discusses ways of using a single panoramic image (captured by a rotating sensor-line camera having very-high spatial resolution) for the geometric shape recovery of a shown object. The objective is to create a sparse polyhedral model, only allowing a few interactive user inputs for a given single panoramic image. The study was motivated by the general question whether a single panoramic image projection allows some kind of 3D shape recovery, possibly benefitting from available monocular approaches for standard (say, pinhole-type) camera models.

Multi-scale 3d-modeling

This paper reviews 3D-modeling activities at the German DLR Institute of Robotics and Mechatronics, carried out within the last decade in cooperation with partners in Germany (Z+F, Illustrated Architecture, DLR Institutes of Optical Information Systems, and of Planetary Research) and international partners. The main focus is on multisensory (e.g. push-broom or rotating stereo line cameras, laser range finders) information containing (at least) geometry and texture. The paper describes systems which acquire such information at different scales of scenery, ranging from indoor scenes to planetary explorations. It also covers principles and methods for preprocessing, geometric reconstruction, texture mapping, or matching.

Stereo Reconstruction from Polycentric Panoramas

The paper defines polycentric panoramas as a generalized model of panoramic images which covers a wide range of previously introduced models such as single-center panoramas, multiperspective panoramic images, or concentric panoramic images. This paper presents geometric fundamentals towards the stereo reconstruction of scenes or objects based on captured pairs of polycentric panoramas. The paper discusses the image acquisition model, epipolar geometry and basics of correspondence analysis. The derived general epipolar curve theorem holds for pairs of polycentric panoramas. This is briefly illustrated by adjusting the general formula of the theorem to specific panoramic image pairs. Experimental results (for two different types of polycentric panoramas) towards stereo reconstruction also demonstrate the practical relevance of the derived geometric fundamentals.