Marco Mammarella

Machine Vision/GPS Integration Using EKF for the UAV Aerial Refueling Problem

The purpose of this paper is to propose the application of an extended Kalman filter (EKF) for the sensors fusion task within the problem of aerial refueling for unmanned aerial vehicles (UAVs). Specifically, the EKF is used to combine the position data from a global positioning system (GPS) and a machine vision (MV)-based system for providing a reliable estimation of the tanker-UAV relative position throughout the docking and the refueling phase. The performance of the scheme has been evaluated using a virtual environment specifically developed for the study of the UAV aerial refueling problem. Particularly, the EKF-based sensor fusion scheme integrates GPS data with MV-based estimates of the tanker-UAV position derived through a combination of feature extraction, feature classification, and pose estimation algorithms. The achieved results indicate that the accuracy of the relative position using GPS or MV estimates can be improved by at least one order of magnitude with the use of EKF in lieu of other sensor fusion techniques.

Comparing Optical Flow Algorithms Using 6-DOF Motion of Real-World Rigid Objects

The application of optical flow algorithms to guidance and navigation problems has gained considerable interest in recent years. This paper summarizes the results of a comparative study on the accuracy of nine different optical flow (OF) algorithms using videos that are captured from an on-board camera during the flight of an autonomous aircraft model. The comparison among the algorithms relies on two formulas that are used both to calculate the ideal OF generated by the motion of a rigid body in the camera field of view and to estimate the linear and angular velocity from the OF.

Addressing corner detection issues for machine vision based UAV aerial refueling

This paper describes the results of the analysis of specific ‘corner detection’ algorithms within a Machine Vision approach for the problem of aerial refueling for unmanned aerial vehicles. Specifically, the performances of the SUSAN and the Harris corner detection algorithms have been compared. A critical goal of this study was to evaluate the interface of these feature extraction schemes with the successive detection and labeling, and pose estimation schemes in the overall scheme. Closed-loop simulations were performed using a Simulink®-based simulation environment to reproduce docking maneuvers using the US Air Force refueling boom.

Comparison of point matching algorithms for the UAV aerial refueling problem

This paper describes the results of a study focused on the evaluation of the performance of specific algorithms for the “point matching” task within the general problem of applying machine vision-based control laws for the problem of aerial refueling (AR) for UAVs. Two different point matching algorithms for the identification of the corner-points of the tanker are proposed. A detailed study of the algorithms is performed with special emphasis on the correct matching and required computational effort. The results show the importance of a correct point-matching scheme for obtaining accurate pose estimation; furthermore, the analysis highlights the tradeoffs involved in the selection of the appropriate algorithms.

A Comparison of Optical Flow algorithms for Real Time Aircraft Guidance and Navigation

This paper focuses on the analysis of the performance of several optical flow algorithms for application within aeronautic applications such as obstacle detection and collision avoidance for lightweight unmanned aerial vehicles (UAVs). The study involved the comparison of nine different optical flow algorithms. Some of these algorithms were developed at West Virginia University (WVU); some were included in the Video and Image Processing Blockset ® of Matlab ® while others were available for download from different sources. The comparative study was performed using both a number of different real-world videos with images rotating or translating at known speeds and a Virtual Reality Environment (VRE) where an aircraft performs complex maneuvers into a detailed virtual world. The comparison of the Optical Flow (OF) algorithms is provided in terms of accuracy of estimation, and an analysis of the computational effort required by the different algorithms was also performed. As expected, algorithms belonging to common conceptual classes turn out to have similar strengths and weaknesses. However, when the computational requirements are taken into account, there is no clear “winning” algorithm, therefore suggesting that ultimately the selection of the “best” algorithm is has to be driven by the particular application.

GPS / MV based Aerial Refueling for UAVs

This paper describes the design of a simulation environment for a GPS / Machine Vision (MV)-based approach for the problem of Aerial Refueling (AR) for Unmanned Aerial Vehicles (UAVs) using the USAF refueling method. MV-based algorithms are implemented within this effort as smart sensor in order to detect the relative position and orientation between the UAV and the tanker. Within this effort, techniques and algorithms for the visualization the tanker aircraft in a Virtual Reality (VR) setting, for the acquisition of the tanker image, for the Feature Extraction (FE) from the acquired image, for the Point Matching (PM) of the features, for the tanker-UAV Pose Estimation (PE) have been developed and extensively tested in closed loop simulations. Detailed mathematical models of the tanker and UAV dynamics, refueling boom, turbulence, wind gusts, and tanker’s wake effects, along with the UAV docking control laws and reference path generation have been implemented within the simulation environment. Mathematical model of the noise produced by GPS, MV, INS and pressure sensors are also derived. This paper also presents an Extended Kalman Filter (EKF) used for the sensors fusion between GPS and MV systems. Results on the accuracy reached for the estimation of the relative position are also provided.

Bounding set calculation for neural network-based output feedback adaptive control systems

Evaluating the bounding set of dynamic systems subject to direct neural-adaptive control is a critical issue in applications where the control system must undergo a rigorous verification process in order to comply with certification standards. In this paper, the boundedness problem is addressed for a comprehensive class of uncertain dynamic systems. Several common but unnecessary approximations that are typically performed to simplify the Lyapunov analysis have been avoided in this effort. This leads to a more accurate and general formulation of the bounding set for the overall closed loop system. The conditions under which boundedness can be guaranteed are carefully analyzed; additionally, the interactions between the control design parameters, the ‘Strictly Positive Realness’ condition, and the shape and dimensions of the bounding set are discussed. Finally, an example is presented in which the bounding set is calculated for the neuro-adaptive control of an F/A-18 aircraft, along with a numerical study to evaluate the effect of several design parameters.

Machine Vision Algorithms for Autonomous Aerial Refueling for UAVs Using the USAF Refueling Boom Method

The purpose of this chapter is to provide an extensive review of a research effort by a team of researchers from West Virginia University and the University of Perugia focused on the design of a Machine Vision (MV)-based system for the Autonomous Aerial Refueling of Unmanned Aerial Vehicles (UAVs) using the US Air Force refueling boom set-up (as opposed to the probe-drogue system used by the US Navy). Following an “Introduction” section with a description of the UAV aerial refueling problem, another section provides an overview of a detailed Simulink-based simulation environment specifically developed for reproducing the UAV/tanker docking maneuver. Next, a section describes the specific approach followed in this effort based on breaking down the problem in a sequence of a Feature Extraction (FE) task (for the purpose of detecting the corners of the tanker from the images on the UAV camera), Detection and Labeling (DAL) task (for the purpose of introducing a tracking for specific corners during the docking), and Pose Estimation (PE), for the purpose of estimating the tanker-UAV relative position during the docking phase. The methodology has been labeled as the FEDALPE approach. The following sections – relative to the Feature Extraction, the Detection and Labeling, and the Pose Estimation - provide comparative studies for a number of methods for each of the above tasks, leading to the selection of the method with the best performance. Another section highlights the advantages of introducing a sensor fusion scheme blending GPS and Machine Vision data for improving the docking performance. A final section summarizes the document providing general conclusion.