Yoram Gat

Video stabilization to a global 3D frame of reference by fusing orientation sensor and image alignment data

Estimating the 3D orientation of the camera in a video sequence within a global frame of reference is useful for video stabilization when displaying the video in a virtual 3D environment, as well as for accurate navigation and other applications. This task requires the input of orientation sensors attached to the camera to provide absolute 3D orientation in a geographical frame of reference. However, high-frequency noise in the sensor readings makes it impossible to achieve accurate orientation estimates required for visually stable presentation of video sequences that were acquired with a camera subject to jitter, such as a handheld camera or a vehicle mounted camera. On the other hand, image alignment has proven successful for image stabilization, providing accurate frame-to-frame orientation estimates but drifting over time due to error and bias accumulation and lacking absolute orientation. In this paper we propose a practical method for generating high accuracy estimates of the 3D orientation of the camera within a global frame of reference by fusing orientation estimates from an efficient image-based alignment method, and the estimates from an orientation sensor, overcoming the limitations of the component methods.

A branch-and-bound technique for nano-structure image segmentation

Images of nano-structures are often noisy. On the other hand, in many settings there is quite a lot of model knowledge regarding the observed structures. This paper proposes a method for segmenting an image using a geometric model of the the observed structure. The resulting segmentation is guaranteed to be globally optimal, for an explicitly specified score function. This property provides a great deal of robustness to the algorithm. The algorithm presented explores a pre-defined space of segmentations using a branch-and-bound algorithm. It eliminates those parts of the space that are provably poor and explores in further detail the more promising parts of the space. An example of a segmentation that can be obtained in this way is a straight line segmentation of an image into 2 regions that minimizes the intensity variation within the regions. Results showing extraction of specific nano-structures are presented. A trivial variation on the algorithm can find a maximum a-posteriori probability estimate of the segmentation when there exists an a-priori distribution over the segmentations and the objective function is interpreted as the likelihood of the image given the segmentation.

Fusing image data with location and orientation sensor data streams for consumer video applications

We discuss the problem of fusing the information in a video stream with synchronized streams of location and orientation data obtained from sensors attached to the video camera. We are interested in using this information for the reconstruction of camera trajectory and observed scenery from consumer videos with the objective of visualizing those in a 3D virtual environment. We review existing literature and applications and suggest application scenarios for personal video. We discuss the issues that distinguish the personal video application from similar applications and present outlines of algorithms for several consumer video application scenarios.

A rigorous lower confidence bound for the expectation of a positive random variable

Given an IID sample from a positive distribution, we provide a method for constructing rigorous finite sample lower confidence bounds for the expectation of the distribution. The method is based on constructing rigorous confidence regions for the cdf of the distribution. We provide some analysis of the asymptotical behavior of the rigorous LCBs. We apply the method to obtain an LCB for a particular, controversial, empirical data set, where the validity of standard methods has been called into question.