Ronald Fevig

Super-resolution mosaicking of UAV surveillance video

This paper explains and implements our efficient multi-frame super-resolution mosaicking algorithm. In this algorithm, feature points between images are matched using SIFT, and then random M-least squares is used to estimate the homography between frames. Next, separate frames are registered and the overlapping region is extracted. A generative model is then adopted and combined with maximum a posteriori estimation to construct the underdetermined sparse linear system. To solve the ill-posed large-scale inverse system, we derive and implement a new hybrid regularization (bilateral total variance Hubert) method. Cross validation is utilized to estimate the derivative of the blur kernel as well as the regularization parameter. Super-resolution is then applied to the individual sub-frames from the overlapping region. Finally, multi-band blending is used to stitch these resolution-enhanced frames to form the final image. The whole process is semi-real time (roughly 30 seconds for 35 frames) and the effectiveness of our algorithm is validated by applying it to real and synthetic UAV video frames.

Image Mosaicking from Uncooled Thermal IR Video Captured by a Small UAV

This paper focuses on a practical technique for mosaicking video frames captured by thermal infrared (IR) cameras flown on a small Unmanned Aerial Vehicle (UAV). A Scale Invariant Feature Transform (SIFT) algorithm is used for detecting the matching feature points. Then, the k-d tree and RANSAC algorithms are used to find the best match as well as to eliminate the outliers. We propose a novel method called random M-least square to find the optimized projective transformation parameters between frames. Finally, we warp the images and adopt the multi-resolution blending method to stitch the registered frames. The whole process is applied to real UAV IR video to validate its robustness to noise and de-focusing. Also, the computational efficiency shows this method is a step in the direction of implementing a real-time UAV system.

GPU-CPU implementation for super-resolution mosaicking of Unmanned Aircraft System (UAS) surveillance video

Unmanned Aircraft Systems (UAS) have been used in many military and civilian applications, particularly surveillance. One of the best ways to use the capacity of a UAS imaging system is by constructing a mosaic of the recorded video. In this paper, we present a novel algorithm to calculate a super-resolution mosaic for UAS, which is both fast and robust. In this algorithm, the features points between frames are found using SIFT (Scale-Invariant Feature Transform), and then RANSAC (Random Sample Consensus) is used to estimate the homography between two consecutive frames. Next, a low-resolution (LR) mosaic is computed. LR images are extracted from the LR mosaic, and then they are subtracted from the input frames to form LR error images. These images are used to compute an error mosaic. The regularization technique uses Huber prior information and is added to the error mosaic to form the superresolution (SR) mosaic. The proposed algorithm was implemented using both a GPU (Graphics Processing Unit) and a CPU (Central Processing Unit). The first part of the algorithm, which is the construction of the LR mosaic, is performed by the GPU, and the rest is performed by the CPU. As a result, there is a significant speed-up of the algorithm. The proposed algorithm has been tested in both the infrared (IR) and visible spectra, using real and synthetic data. The results for all these cases show a great improvement in resolution, with a PSNR of 41.10 dB for synthetic data, and greater visual detail for the real UAV surveillance data.

Hierarchical multi-level image mosaicing for autonomous navigation of UAV

A novel algorithm for hierarchical multi-level image mosaicing for autonomous navigation of UAV is proposed. The main contribution of the proposed system is the blocking of the error accumulation propagated along the frames, by incrementally building a long-duration mosaic on the fly which is hierarchically composed of short-duration mosaics. The proposed algorithm fulfills the real-time processing requirements in autonomous navigation as follows. 1) Causality: the current output of the mosaicing system depends only on the current and/or previous input frames, contrary to existing offline mosaic algorithms that depend on future input frames as well. 2) Learnability: the algorithm autonomously analyzes/learns the scene characteristics. 3) Adaptability: the system automatically adapts itself to the scene change and chooses the proper methods for feature selection (i.e., the fast but unreliable LKT vs. the slow but robust SIFT). The evaluation of our algorithm with the extensive field test data involving several thousand airborne images shows the significant improvement in processing time, robustness and accuracy of the proposed algorithm.

Robust Spatial-Domain Based Super-Resolution Mosaicing of CubeSat Video Frames: Algorithm and Evaluation

In this paper, a spatial domain based super-resolution mosaicing system and its evaluation results are presented. This algorithm incorporates two main algorithms, an image mosaicing algorithm and a super-resolution-reconstruction algorithm. Huber-based prior information is used to address the ill-posed nature of the super resolution problem. To test the efficiency of the proposed algorithm, four performance metrics are used: mean square error, peak signal-to-noise ratio, singular value decomposition based measure, and cumulative probability of blur detection. Testing is performed using datasets generated from both a high altitude balloon and an Unmanned Aerial Vehicle (UAV). Results show that the proposed algorithm is highly efficient in real-time applications, such as remote sensing on a small satellite (i.e., CubeSat.) Furthermore, the performance metrics are proven to be accurate in quantitative evaluation.

Earth impactors: threat analysis and multistage intervention mission architecture

Earth impactors (EIs) pose a significant threat. Upon EI detection, a response mission is required. The proposed architecture is suitable for responding to 75% of EIs. For rapid response, the reconnaissance and the tactical nuclear intervener craft are launched in close succession. The extended response timeframe allows collected data analysis before launching an intervener craft to slowly shift the EIs orbit. A small spacecraft equipped with a radio science package, visual camera, multi-spectral imager, LIDAR and, optionally, a radar tomography sensor will be used for reconnaissance. Sensor tasking and control will be autonomous based on controller-supplied objectives.

Sensor fusion method using GPS/IMU data for fast UAV surveillance video frame registration

This paper proposes an innovative framework for fast image registration of UAV surveillance video frames by fusing the data from a GPS receiver high-frequency IMU sensor (Piccolo autopilot) and a feature-domain registration method through a non-linear filter. The high-frequency imprecise data from the Piccolo autopilot is refined by the low-frequency precise data from our feature-domain based random M least squares (RMLS) method. The projective transformation model is chosen to achieve high precision. The state and measurement models are non-linear to approximate the real-world imaging dynamics. A periodic hybrid particle filter (PHPF), composed of extended Kalman filter (EKF) and unscented Kalman filter (UKF), is proposed to minimize running time while maintaining accuracy. Both the efficiency and effectiveness of the proposed algorithm will be evaluated through our experiments.

Orbit Stability Determination of Satellites Using Harmonic Force Excitation Analysis

The focus of this paper is the determination of orbit stability of a small satellite around an asteroid with a complex gravitational field using harmonic force excitation analysis. The determination of stable orbits is critical in space mission planning, especially in deep space missions investigating asteroids having undetermined mass distribution. Many of these asteroids have complex gravitational fields that make close orbits, which are necessary for inspection, very difficult to predict and safely maintain. Simulations of the asteroid Itokawa, which was visited and analyzed by the Hayabusa Space Mission, have shown such complex gravitational fields. Orbit simulations using Systems Tool Kit (STK) software have demonstrated many interesting phenomena resulting from the nonlinear satellite/asteroid interaction. One behavior of special note was the influence of the rate of spin of the asteroid and the stability of the orbit. This behavior was found to be linked to the frequency and magnitude of the gravitational excitation force along with altitude of the orbiting satellite. A harmonic force excitation analysis of the system was shown to be an accurate predictor of orbital stability. The resulting frequency ratio determined is shown to predict regions where complex gravity effects are significant.

Harmonic Force Excitation Analysis of a Small-Body Asteroid/Satellite System

A harmonic force excitation analysis is used to determine orbit stability of a small satellite around an asteroid with a complex gravitational field. Harmonic excitation phenomena occurs with both natural and man-made satellites. Jupiter influences asteroid distribution in the main asteroid belt through mean motion resonance where some regions are devoid of asteroids while other regions have an abundance. Simulations of a man-made satellite in orbit around the asteroid Itokawa, which was visited by the Hayabusa Space Mission, have also displayed harmonic excitation phenomena, including regions of high dynamic interactions. Specifically, the influence of the rate of spin of the asteroid and the stability of the orbit was investigated. The radial acceleration of the satellite is used to determine the frequency of gravitational perturbation from the asteroid on the satellite. It has been shown when the satellite is placed in an orbit away from its resonant frequency, the orbit remains stable. An early model was created to study this phenomenon and showed promise to predict regions of stability. From this initial study, further work using a more complex model and an updated harmonic force excitation analysis of the system was shown to be a more accurate predictor of orbital stability.

Panorama recovery from noisy UAV surveillance video

This paper proposes an efficient and robust algorithm to recover a panorama from poorly-obtained UAV video frames contaminated with significant noise. In this algorithm, the eigen-space based neighborhood region will be introduced with our novel feature-based random M least-sqaures (RMLS) registration technique. Meanwhile, the corresponding similarity regions will be assigned weights according to the relativity between these neighboring regions. Next, Bayesian multi-frame sampling will be implemented utilizing the homography estimated by the frame registration. Finally, the sub-region in each frame which is applicable to the multi-frame sampling will be stitched utilizing multi-resolution blending.