Xiaochun Cao

Video Completion for Perspective Camera Under Constrained Motion

This paper presents a novel technique to fill in missing background and moving foreground of a video captured by a static or moving camera. Different from previous efforts which are typically based on processing in the 3D data volume, we slice the volume along the motion manifold of the moving object, and therefore reduce the search space from 3D to 2D, while still preserve the spatial and temporal coherence. In addition to the computational efficiency, based on geometric video analysis, the proposed approach is also able to handle real videos under perspective distortion, as well as common camera motions, such as panning, tilting, and zooming. The experimental results demonstrate that our algorithm performs comparably to 3D search based methods, and however extends the current state-of-the-art repairing techniques to videos with projective effects, as well as illumination changes

Camera calibration and light source orientation from solar shadows

In this paper, we describe a method for recovering camera parameters from perspective views of daylight shadows in a scene, given only minimal geometric information determined from the images. This minimal information consists of two 3D stationary points and their cast shadows on the ground plane. We show that this information captured in two views is sufficient to determine the focal length, the aspect ratio, and the principal point of a pinhole camera with fixed intrinsic parameters. In addition, we are also able to compute the orientation of the light source. Our method is based on exploiting novel inter-image constraints on the image of the absolute conic and the physical properties of solar shadows. Compared to the traditional methods that require images of some precisely machined calibration patterns, our method uses cast shadows by the sun, which are common in natural environments, and requires no measurements of any distance or angle in the 3D world. To demonstrate the accuracy of the proposed algorithm and its utility, we present the results on both synthetic and real images, and apply the method to an image-based rendering problem.

Camera calibration and light source estimation from images with shadows

In this paper, we describe how camera parameters and light source orientation can be recovered from two perspective views of a scene given only two vertical lines and their cast shadows. Compared to the traditional calibration methods that involve images of some precisely machined calibration pattern, our method uses new calibration objects: the vertical objects and their parallel shadow lines, which are common in natural environments. In addition to the benefit of increasing accessibility of the calibration objects, the proposed method is also especially useful in cases where only limited information is available. To demonstrate the accuracy and the applications of the proposed algorithm, we present results on both synthetic and real images.

Camera Calibration Using Symmetric Objects

This paper proposes a novel method for camera calibration using images of a mirror symmetric object. Assuming unit aspect ratio and zero skew, we show that interimage homographies can be expressed as a function of only the principal point. By minimizing symmetric transfer errors, we thus obtain an accurate solution for the camera parameters. We also extend our approach to a calibration technique using images of a 1-D object with a fixed pivoting point. Unlike existing methods that rely on orthogonality or pole-polar relationship, our approach utilizes new interimage constraints and does not require knowledge of the 3-D coordinates of feature points. To demonstrate the effectiveness of the approach, we present results for both synthetic and real images

Self-calibration from turn-table sequences in presence of zoom and focus

This paper proposes a novel method, using constant inter-frame motion, for self-calibration from an image sequence of an object rotating around a single axis with varying camera internal parameters. Our approach makes use of the facts that in many commercial systems rotation angles are often controlled by an electromechanical system, and that the inter-frame essential matrices are invariant if the rotation angles are constant but not necessary known. Therefore, recovering camera internal parameters is possible by making use of the equivalence of essential matrices which relate the unknown calibration matrices to the fundamental matrices computed from the point correspondences. We also describe a linear method that works under restrictive conditions on camera internal parameters, the solution of which can be used as the starting point of the iterative non-linear method with looser constraints. The results are refined by enforcing the global constraints that the projected trajectory of any 3D point should be a conic after compensating for the focusing and zooming effects. Finally, using the bundle adjustment method tailored to the special case, i.e., static camera and constant object rotation, the 3D structure of the object is recovered and the camera parameters are further refined simultaneously. To determine the accuracy and the robustness of the proposed algorithm, we present the results on both synthetic and real sequences.

Single view compositing with shadows

In this paper, we describe how geometrically correct and visually realistic shadows may be computed for objects composited into a single view of a target scene. Compared to traditional single view compositing methods, which either do not deal with the shadow effects or manually create the shadows for the composited objects, our approach efficiently utilizes the geometric and photometric constraints extracted from a single target image to synthesize the shadows consistent with the overall target scene for the inserted objects. In particular, we explore (i) the constraints provided by imaged scene structure, e.g. vanishing points of orthogonal directions, for camera calibration and thus explicit determination of the locations of the camera and the light source; (ii) the relatively weaker geometric constraint, the planar homology, that models the imaged shadow relations when explicit camera calibration is not possible; and (iii) the photometric constraints that are required to match the color characteristics of the synthesized shadows with those of the original scene. For each constraint, we demonstrate the working examples followed by our observations. To show the accuracy and the applications of the proposed method, we present the results for a variety of target scenes, including footage from commercial Hollywood movies and 3D video games.

Camera calibration without metric information using 1D objects

This paper addresses the problem of calibrating a pin-hole camera from images of 1D objects. Assuming a unit aspect ratio and zero skew, we introduce a novel and simple approach that uses four observations of a 1D object and requires no information about the distances between the points on the object. This is in contrast to existing methods that use two images, but impose more restrictive configurations that require measured distances on the calibrating object. The key features of the proposed technique are its simplicity and ease of use due to the lack of need for any metric information. To demonstrate the effectiveness of the algorithm, we present the processing results on synthetic and real images.

Camera Calibration from Two Shadow Trajectories

We introduce an efficient method for recovering the camera parameters automatically from the cast shadows of two 3D points observed over time. Compared to previous related work, our method has less restrictions in the sense that object-to-shadow correspondences do not have to be available in the image. We demonstrate how the horizon line may be recovered from only shadow points, and how the camera intrinsic and extrinsic parameters are determined using the pole-polar relationship and minimizing the algebraic distance of the principal point. The approach is fully validated on both synthetic and real data, and tested against various sources of error. We finally present an application to metrology from shadows only - i.e. when the object is not visible in the image

Simple calibration without metric information using an isoceles trapezoid

This paper addresses the problem of calibrating a pin-hole camera from images of an isoceles trapezoid. Assuming a unit aspect ratio and zero skew, we introduce a novel and simple camera calibration approach. The key features of the proposed technique are its simplicity and the lack of need for 3D coordinate information about the calibrating object - i.e. the isosceles trapezoid. By utilizing the symmetry of such trapezoid, we show that one can obtain both the internal and the external camera parameters. To demonstrate the effectiveness of the algorithm, we present the processing results on synthetic and real images, and compare our results to Zhangs flexible calibration method.

Metrology in uncalibrated images given one vanishing point

In this paper, we describe how 3D Euclidean measurements can be made in a pair of uncalibrated images, when only minimal geometric information are available in the image planes. This minimal information consists of a line in a reference plane, and the vanishing point orthogonal to it. Given such limited information, we show that the length ratio of two objects perpendicular to the reference plane can be expressed as a function of the camera intrinsic parameters. Assuming that the camera intrinsic parameters remain invariant between two views, we perform Euclidean metric measurements directly in the perspective images.