Wenyi Zhao

Is Super-Resolution with Optical Flow Feasible?

Reconstruction-based super-resolution from motion video has been an active area of study in computer vision and video analysis. Image alignment is a key component of super-resolution algorithms. Almost all previous super-resolution algorithms have assumed that standard methods of image alignment can provide accurate enough alignment for creating super-resolution images. However, a systematic study of the demands on accuracy of multi-image alignment and its effects on super-resolution has been lacking. Furthermore, implicitly or explicitly most algorithms have assumed that the multiple video frames or specific regions of interest are related through global parametric transformations. From previous works, it is not at all clear how super-resolution performs under alignment with piecewise parametric or local optical flow based methods. This paper is an attempt at understanding the influence of image alignment and warping errors on super-resolution. Requirements on the consistency of optical flow across multiple images are studied and it is shown that errors resulting from traditional flow algorithms may render super-resolution infeasible.

Video georegistration: algorithm and quantitative evaluation

An algorithm is presented for video georegistration, with a particular concern for aerial video, i.e., video captured from an airborne platform. The algorithms input is a video stream with telemetry (camera model specification sufficient to define an initial estimate of the view) and geodetically calibrated reference imagery (coaligned digital orthoimage and elevation map). The output is a spatial registration of the video to the reference so that it inherits the available geodetic coordinates. The video is processed in a continuous fashion to yield a corresponding stream of georegistered results. Quantitative results of evaluating the developed approach with real world aerial video also are presented. The results suggest that the developed approach may provide valuable input to the analysis and interpretation of aerial video.

Super-fusion: a super-resolution method based on fusion

Reconstruction-based super-resolution algorithms require very accurate alignment and good choice of filters to be effective. Often these requirements are hard to satisfy, for example, when we adopt optical flow as the motion model. In addition, the condition of having enough sub-samples may vary from pixel to pixel. We propose an alternative super-resolution method based on image fusion (called super-fusion hereafter). Image fusion has been proven to be effective in many applications. Extending image fusion to super-resolve images, we show that super-fusion is a faster alternative that imposes less requirements and is more stable than traditional super-resolution methods.

Video enhancement by scintillation removal

As a comprehensive problem, video enhancement includes many subset problems, including super-resolution, noise reduction, increase of dynamic range, to name a few. In this paper, we focus on a special subset problem - scintillation removal. Scintillation is one type of image distortions caused by the propagation of visible light through unstable atmosphere. One commonly known source for disturbing atmosphere is heat. We propose an image processing method to remove scintillation in video. This method explores the temporal information redundancy in video and is based on image registration and image fusion. We examine the scintillation problem in detail and explain why we take an image registration plus fusion approach. Finally we demonstrate successful application of scintillation removal on real video clips.