Shih-Schon Lin

High resolution catadioptric omni-directional stereo sensor for robot vision

Autonomous robots need to acquire both omnidirectional view and stereo in real-time without sacrificing too much image resolution. The recent catadioptric omni-directional vision sensors provide simple affordable real-time omni-directional images but only at the cost of rather low image resolutions. This work describes a novel omni-directional stereo image sensor providing the highest resolution while still retaining the simplicity and real-time advantages that are unique to the family of catadioptric sensors. Only one simple reflective surface, a beam splitter and two regular (perspective) cameras are needed.

True single view point cone mirror omni-directional catadioptric system

Pinhole camera model is a simplified subset of geometric optics. In special cases like the image formation of the cone (a degenerate conic section) mirror in an omnidirectional view catadioptric system, there are more complex optical phenomena involved that the simple pinhole model can not explain. We show that using the full geometric optics model a true single viewpoint cone mirror omni-directional system can be built. We show how such a system is built first, and then show in detail how each optical phenomenon works together to make the system true single viewpoint. The new system requires only simple off-the-shelf components and still outperforms other single viewpoint omni-systems for many applications.

Single-view-point omnidirectional catadioptric cone mirror imager

We present here a comprehensive imaging theory about cone mirrors in a single-view-point (SVP) configuration and show that an SVP cone mirror catadioptric system is not only practical but also has unique advantages for certain applications. We show its merits and weaknesses and how to build a workable system.

Single-viewpoint, catadioptric cone mirror omnidirectional imaging theory and analysis

A family of catadioptric imaging systems has been developed that can achieve omnidirectional viewing with a single planar imager while still being able to recover perspective images, provided that they satisfy the single-viewpoint (SVP) constraint. It has been shown that the only mirror shapes that can have SVP when paired with a sole focusing planar imager camera are the surfaces of revolution of conic section curves. However, the special case of such a surface, the cone-shaped mirror itself, has not been deemed a viable SVP mirror shape. We present a comprehensive imaging theory of the cone mirror in its SVP configuration. We show that the SVP, cone mirror catadioptric system not only is practical but also has unique advantages for certain applications. The detailed theory explains why and how a practical SVP cone configuration can be set up, the merits and weaknesses of such systems, and how one can remedy the weaknesses to create a workable imaging system. We also derive the tolerance formula for estimating effects of alignment errors. A prototype has been constructed, and experimental results validate our theory.

Bio-inspired visualization of polarization information using temporal fusion, flicker, coherently moving dots and texture

The human eye is effectively “polarization-blind”, i.e., it is only sensitive to the intensity and wavelength of optical waves, and encodes and maps these into the perceptual qualities of brightness and color. Polarization, the third characteristic of optical waves, however, has been shown to be an important information-carrying features of waves, and may reveal useful information about a scene such as local curvature, material contrast, surface shapes, and the relative direction of illumination [1]. Since the unaided human eye cannot sense the polarization of light, information collected by polarimetric imaging systems must be exh ibited as a set of cues visible to a human observer, preferably in a way that, by and large, preserves important non-polarization information such as that carried by the spectral and luminance distributions in the image. Effectively, some form of “sensory substitution” is required for presenting polarization “signals” to a “polarization-blind” observer, without affecting other visual information such as color and brightness.

Biomimetic, adaptive, optimum polarization-opponent imaging of scenes with preferential polarization distributions

Certain animal species have visual systems that are sensitive to light’s polarization [1-3]. The human eye, on the other hand, is not capable of “seeing” polarization information, without any instrument. However, man-made imaging systems can collect polarization information from a scene, in addition to luminance and spectral information. One of the important issues in polarization imaging is how to process the polarization information after (or while) it is collected by the imaging system. As one possible processing technique, in our earlier work we introduced the methods of “polarization-difference imaging” (PDI), inspired by polarization vision in certain species, and we demonstrated that optical imaging systems utilizing PDI techniques may facilitate the detection of targets in scattering media, even when the targets produce only very weak polarization, and that such enhancement can increase the distance over which targets can be detected [4-6]. In that work, we showed that in the PDI technique, the intensities of the two orthogonal polarization components of imaging-forming light for each pixel at (x, y), i.e., I║ (x, y) and I⊥ (x, y), were captured and then the “polarization-sum” (PS) and “polarization difference” (PD) intensities were formed as IPS (x, y) =I║ (x, y) +I⊥ (x, y) and IPS (x, y) = I║ (x, y)– I⊥ (x, y) Furthermore, it was shown by Tyo [7] that the optimum linear combination of I║ (x, y) and I⊥ (x, y) channels for a scene, in which the polarization angle is assumed to be a random variable with a uniform probability density function, are indeed PS and PD signals, utilizing the analogy and parallelism with the biology of color vision in the human visual system and related principle component analysis done by Buchsbaum and Gottschalk [8]. Therefore, in that scenario the PS and PD channels were optimum in the information theoretic sense, i.e., their contents were statistically uncorrelated, requiring minimum bandwidth [7]. However, the situation may differ when the scene statistically has a preferential polarization distribution with a non-uniform probability density function for the state of polarization. What would then be the optimum linear combination for polarization channels in this case? We have been investigating techniques to adaptively form such optimum linear combination for polarization channels. Specifically, the two linear channels in general can be expressed as I P S a d a p t i v e (x, y) =a I║ (x, y) + β I⊥ (x, y) and I P D a d a p t i v e (x, y) = γ I║ (x, y) + ζ I⊥ (x, y) with unequal weighting coefficients a, b, g, and z, which should be determined based on the statistics of the polarization distribution in any given scene. Utilizing the technique of principal components analysis for non-uniform distributions of polarization state in an image, we determine the appropriate values for these coefficients for forming the optimum linear combination of polarization channels. These coefficients, therefore, depend on the polarization statistics of the scene, and can be adaptively adjusted as the imaging system observes different environments. Such optimum combinations for polarization channels with unequal weighting coefficients suitable for environment with preferential polarization distribution (such as under water) may point to an interesting processing in the polarization vision in certain aquatic species, and can lead to images with higher contrast and better target detection for man-made imaging systems.