Christoforos Kanellakis

Survey on Computer Vision for UAVs: Current Developments and Trends

During last decade the scientific research on Unmanned Aerial Vehicless (UAVs) increased spectacularly and led to the design of multiple types of aerial platforms. The major challenge today is the development of autonomously operating aerial agents capable of completing missions independently of human interaction. To this extent, visual sensing techniques have been integrated in the control pipeline of the UAVs in order to enhance their navigation and guidance skills. The aim of this article is to present a comprehensive literature review on vision based applications for UAVs focusing mainly on current developments and trends. These applications are sorted in different categories according to the research topics among various research groups. More specifically vision based position-attitude control, pose estimation and mapping, obstacle detection as well as target tracking are the identified components towards autonomous agents. Aerial platforms could reach greater level of autonomy by integrating all these technologies onboard. Additionally, throughout this article the concept of fusion multiple sensors is highlighted, while an overview on the challenges addressed and future trends in autonomous agent development will be also provided.

On the design, modeling and control of a novel compact aerial manipulator

The aim of this article is to present a novel four-degree-of-freedom aerial manipulator allowing a multirotor Unmanned Aerial Vehicle (UAV) to physically interact with the environment. The proposed design, named CARMA (Compact AeRial MAnipulator), is characterized by low disturbances on the UAV flight dynamics, extended workspace (with regard to its retracted configuration) and fast dynamics (compared to the UAV dynamics). The dynamic model is formulated and a control structure consisting of an inverse kinematics algorithm and independent joint position controllers is presented. Furthermore, the design specifications of the prototype are analyzed in detail, while experimental evaluations are conducted for the extraction of the manipulators workspace and the evaluation of systems tracking capabilities over pick-and-place trajectories. Finally, it is shown that the selected joint position sensors, combined with the derived inverse dynamic algorithm allow to determine the wrenches exerted at the base, due to swift motions of the arm.

Cooperative UAVs as a Tool for Aerial Inspection of the Aging Infrastructure.

This article presents an aerial tool towards the autonomous cooperative coverage and inspection of a 3D infrastructure using multiple Unmanned Aerial Vehicles (UAVs). In the presented approach the UAVs are relying only on their onboard computer and sensory system, deployed for inspection of the 3D structure. In this application each agent covers a different part of the scene autonomously, while avoiding collisions. The visual information collected from the aerial team is collaboratively processed to create the 3D model. The performance of the overall setup has been experimentally evaluated in a realistic outdoor infrastructure inspection experiments, providing sparse and dense 3D reconstruction of the inspected structures.

Dynamic visual sensing based on MPC controlled UAVs

This article considers the establishment of a dynamic visual sensor from monocular cameras to enable a reconfigurable environmental perception. The cameras are mounted on Micro Aerial Vehicles (MAV) which are coordinated by a Model Predictive Control (MPC) scheme to retain overlapping field of views and form a global sensor with varying baseline. The specific merits of the proposed scheme are: a) the ability to form a configurable stereo rig, according to the application needs, and b) the simple design, the reduction of the payload and the corresponding cost. Moreover, the proposed configurable sensor provides a glpobal 3D reconstruction of the surrounding area, based on a modified Structure from Motion approach. The efficiency of the suggested flexible visual sensor is demonstrated in simulation results that highlight the novel concept of cooperative flying cameras and their 3D reconstruction capabilities.

Reduced complexity calibration of MEMS IMUs

In this article a reduced complexity calibration method for Micro-Electro-Mechanical Systems (MEMS) Inertial Measurement Units (IMUs) will be presented, which does not need the rotating reference tables, commonly used in the gyroscope calibration. As it will be presented, in the proposed novel scheme fixed angle rotations have been utilized to observe the integral of the gyroscope signals to find the corresponding sensitivity, axis misalignment and acceleration sensitivity matrices. This appraoch has the significant merit of high norm accuracy, easiness of use, low cost and simplicity in construction, thus allowing anyone with a basic electronics knowledge to calibrate an IMU.

Cooperative coverage for surveillance of 3D structures

In this article, we propose a planning algorithm for coverage of complex structures with a network of robotic sensing agents, with multi-robot surveillance missions as our main motivating application. The sensors are deployed to monitor the external surface of a 3D structure. The algorithm controls the motion of each sensor so that a measure of the collective coverage attained by the network is nondecreasing, while the sensors converge to an equilibrium configuration. A modified version of the algorithm is also provided to introduce collision avoidance properties. The effectiveness of the algorithm is demonstrated in a simulation and validated experimentally by executing the planned paths on an aerial robot.

Towards Autonomous Surveying of Underground Mine Using MAVs

Micro Aerial Vehicles (MAVs) are platforms that received great attention during the last decade. Recently, the mining industry has been considering the usage of aerial autonomous platforms in their processes. This article initially investigates potential application scenarios for this technology in mining. Moreover, one of the main tasks refer to surveillance and maintenance of infrastructure assets. Employing these robots for underground surveillance processes of areas like shafts, tunnels or large voids after blasting, requires among others the development of elaborate navigation modules. This paper proposes a method to assist the navigation capabilities of MAVs in challenging mine environments, like tunnels and vertical shafts. The proposed method considers the use of Potential Fields method, tailored to implement a sense-and-avoid system using a minimal ultrasound-based sensory system. Simulation results demonstrate the effectiveness of the proposed strategy.

Evaluation of visual localization systems in underground mining

In this article an evaluation of current technology on visual localization systems for underground mining is presented. The proposed study is considered to be the first step among others towards enabling vision-based mine inspection using Unmanned Micro Aerial Vehicles (UAVs). Furthermore, the aim of this article, is to verify applicable and reliable low cost existing methods and technologies for the problem of UAV localization in harsh and challenging environments. More specifically field trials were performed in one of the biggest mines in Europe, the iron ore mine of LKAB at Kiruna in Sweden. In this experimental evaluation the sensors employed were a RGB-D camera (Kinect for Windows) and a web camera (Playstation 3 Eye) in two configurations, as a stereo rig and as a monocular visual sensor. The processing of the stored data from the experiments will provide an insight into the efficiency of these sensors. Additionally, will identify what further technological and research developments are required to develop affordable autonomous UAV solutions for improving the underground mining production tasks.

A robust reconfigurable control scheme against pose estimation induced time delays

Time delays are one of the most common problems when utilizing a visual sensor for pose estimation or navigation in aerial robotics. Such time delays can grow exponentially as a function of the scenes complexity and the size of the mapping during classical Simultaneous Localization and Mapping (SLAM) strategies. In this article, a robust reconfigurable control scheme against pose estimation induced time delays is presented. Initially, an experimental verification of the induced time delays via pose estimation is performed for the attitude problem of a hexacopter, while a switching time delay dependent modeling approach is formulated. In addition, a stability analysis algorithm is introduced in order to evaluate the maximum allowable time delays that the target system can handle for a given LQR controller. The varying nature of the time delays results in a switching system with the latency time to play the role of a switching rule, while simulation results are presented to outline the effects of the time-induced delays in hexarotor-based systems and finally evaluate the overall efficiency of the proposed control scheme.

A low-cost stereoscopic µP-based vision system for industrial light objects grasping

This article describes a vision-based manipulation process for real-time moving objects tracking and grasping, aiming at industrial manufacturing and assembling applications. The adoption of computer vision techniques for object recognition is implemented by means of a stereoscopic system using color based methods under OpenCV libraries. The visual software is directly coupled with the control software of the robotic arm Katana 6M90G manufactured by Neuronics AG, running under a Linux-based Operating System (OS) distribution Lubuntu, over a low-cost and powerful microprocessor Odroid U3. Experimental studies validate the effectiveness of the implementation, while remarking the advantageous effects of the 3D pose estimation process.