Sergei V. Fogel

The estimation of velocity vector fields from time-varying image sequences

Abstract Changes in successive images from a time-varying image sequence of a scene can be characterized by velocity vector fields. The estimate of the velocity vector field is determined as a compromise between optical flow and directional smoothness constraints. The optical flow constraints relate the values of the time-varying image function at the corresponding points of the successive images of the sequence. The directional smoothness constraints relate the values of neighboring velocity vectors. To achieve the compromise, we introduce a system of nonlinear equations of the unknown estimate of the velocity vector field using a novel variational principle applied to the weighted average of the optical flow and the directional smoothness constraints. A stable iterative method for solving this system is developed. The optical flow and the directional smoothness constraints are selectively suppressed in the neighborhoods of the occluding boundaries by implicitly adjusting their weights. These adjustments are based on the spatial variations of the estimates of the velocity vectors and the spatial variations of the time-varying image function. The system of nonlinear equations is defined in terms of the time-varying image function and its derivatives. The initial image functions are in general discontinuous and cannot be directly differentiated. These difficulties are overcome by treating the initial image functions as generalized functions and their derivatives as generalized derivatives. These generalized functions are evaluated (observed) on the parametric family of testing (smoothing) functions to obtain parametric families of secondary images, which are used in the system of nonlinear equations. The parameter specifies the degree of smoothness of each secondary image. The secondary images with progressively higher degrees of smoothness are sampled with progressively lower resolutions. Then coarse-to-fine control strategies are used to obtain the estimate.

Restoration of spatially variant motion blurs in sequential imagery

In many modern applications, the image data consists of a sequence of frames which may be degraded by object motion. Examples include high- speed television where the purpose is to record and analyze rapid motion of objects and production of still frame hardcopy from vide recordings. It is often the case that the motion of the image of the object on the recording sensor moves significantly during the time the shutter is open. Since different objects may be moving in different directions at different speeds, the resulting blur is space-variant. Two problems are addressed in this research: determination of the space-variant motion and restoration with space-variant point spread functions. The work presented here uses the surrounding frames to obtain information about the relative motion, and together with information about the shutter speed, estimates the motion blur in local regions of the image. The advantage is that there are no limitations of the type of motion which can be treated. A restoration method has been developed which can deal with this directly, as opposed to geometrically warping the image to produce an image which is treated by spatially-invariant methods. The method is a modification of the iterative Landweber iteration which is implemented using overlapping sections.