Shaowu Pan

A Kinect-Based Real-Time Compressive Tracking Prototype System for Amphibious Spherical Robots

A visual tracking system is essential as a basis for visual servoing, autonomous navigation, path planning, robot-human interaction and other robotic functions. To execute various tasks in diverse and ever-changing environments, a mobile robot requires high levels of robustness, precision, environmental adaptability and real-time performance of the visual tracking system. In keeping with the application characteristics of our amphibious spherical robot, which was proposed for flexible and economical underwater exploration in 2012, an improved RGB-D visual tracking algorithm is proposed and implemented. Given the limited power source and computational capabilities of mobile robots, compressive tracking (CT), which is the effective and efficient algorithm that was proposed in 2012, was selected as the basis of the proposed algorithm to process colour images. A Kalman filter with a second-order motion model was implemented to predict the state of the target and select candidate patches or samples for the CT tracker. In addition, a variance ratio features shift (VR-V) tracker with a Kalman estimation mechanism was used to process depth images. Using a feedback strategy, the depth tracking results were used to assist the CT tracker in updating classifier parameters at an adaptive rate. In this way, most of the deficiencies of CT, including drift and poor robustness to occlusion and high-speed target motion, were partly solved. To evaluate the proposed algorithm, a Microsoft Kinect sensor, which combines colour and infrared depth cameras, was adopted for use in a prototype of the robotic tracking system. The experimental results with various image sequences demonstrated the effectiveness, robustness and real-time performance of the tracking system.

A Spherical Robot based on all Programmable SoC and 3-D Printing

As the competition and exploration on marine is going more intense, intelligent underwater robots or autonomous underwater vehicles have become important tools to accomplish tasks such as ocean geologic survey, mineral exploration, victims rescue, military reconnaissance, etc. The mechanical structure and electronic control module design, which have long been research hotspots in this field, are in direct relation to the performance, reliability, maintainability and deployment ability of underwater robots. Existing designs commonly develop the body of robots with a large quantity of components or parts and realize information processing and motion control by using multiple kinds of electronic devices. This paper proposed a novel technical solution for underwater robots design. 3-D printing technology was adopted to construct the body of our spherical amphibious robots directly, which eliminated manufacturing difficulty, shortened the production cycle and improved water-tightness. Xilinx Zynq-7000 All Programmable SoC was used to integrate motor control, sensor management, data acquisition and other functional units or circuits into a single chip. Moreover, a universal FMC slot for high speed board and four PMod sockets for low speed modules were provided on the mother board of the electronic control module to expand functions, upgrade hardware or install scientific instruments for special missions. By introducing 3-D printing and latest SoC technologies, the system complexity of the spherical robot was significantly reduced, which improved reliability. Meanwhile, the design room for intelligent robots was broadened with modular design and all programmable SoC. The design in this paper may have reference value for multipurpose underwater robots or vehicles.

Visual Detection and Tracking System for a Spherical Amphibious Robot

With the goal of supporting close-range observation tasks of a spherical amphibious robot, such as ecological observations and intelligent surveillance, a moving target detection and tracking system was designed and implemented in this study. Given the restrictions presented by the amphibious environment and the small-sized spherical amphibious robot, an industrial camera and vision algorithms using adaptive appearance models were adopted to construct the proposed system. To handle the problem of light scattering and absorption in the underwater environment, the multi-scale retinex with color restoration algorithm was used for image enhancement. Given the environmental disturbances in practical amphibious scenarios, the Gaussian mixture model was used to detect moving targets entering the field of view of the robot. A fast compressive tracker with a Kalman prediction mechanism was used to track the specified target. Considering the limited load space and the unique mechanical structure of the robot, the proposed vision system was fabricated with a low power system-on-chip using an asymmetric and heterogeneous computing architecture. Experimental results confirmed the validity and high efficiency of the proposed system. The design presented in this paper is able to meet future demands of spherical amphibious robots in biological monitoring and multi-robot cooperation.

3D Printing Technology-based an Amphibious Spherical Robot

It has long been recognized that the employment of underwater robots have important practical significance, which includes pipe survey, oceanic search, under-ice exploration, mine reconnaissance, dam inspection, ocean survey and so on. Owing to the limitation of underwater environment, some regular sized robots are not suitable for limited spaces. Thus some micro-robots appeared, while sacrificed important abilities such as locomotion velocity and enduring time to achieve compact sizes. Then a mother-son robot system was proposed in our previous researches, which included several micro-robots as sons and an amphibious spherical robot as the mother. The mother robot was adopted to make up for the shortages of microrobots. This paper mainly focused on the structure and mechanism of the mother robot. The mother robot was designed with a spherical structure, which was composed of a fixed hemisphere hull and two operable quarter spherical hulls. It was actuated by four water-jet propellers and ten servomotors, capable of moving on land and in underwater environment. We developed a prototype and evaluated its walking and swimming motions in our previous experiments. Due to some problems in the process of assembly, the motion stability and reliability performed not so well. So, in this paper, we improved the structure and mechanism of the robot based on 3D Printing, which could eliminate some manufacture difficulties, shorten the production cycle, improve water-tightness, and enhance the robots overall stability, compactness and aesthetics.

A low-power SoC-based moving target detection system for amphibious spherical robots

A moving target detection and tracking system is critical important for autonomous mobile robot to accomplish complicated tasks. Aiming at application requirements of our amphibious spherical robot proposed in previous researches, a low-power and portable moving target detection system was designed and implemented in this paper. Xilinx Zynq-7000 SoC (System on Chip) was used to fabricate the image processing system of the robot for detection and tracking. An OmniVision OV7670 COMS image sensor controlled by customized IP cores in the PL (Programmable Logic) of the SoC was adopted to acquire 640×480 RGB images at 30 frames per second. The Gaussian background modeling method was implemented with Vivado HLS in the PL to detect moving targets. And a FCT (Fast Compressive Tracking) tracker with motion estimation mechanism was running in the PS (Processing System) of the SoC to track targets captured by the detection subsystem subsequently. Besides, the dynamical power management (DPM) and the dynamical voltage frequency scaling (DVFS) mechanisms were used for a higher power-efficiency. Experimental results verified the validation and performance of the detection system. The design in this paper may have reference value for vision-based mobile robots or vehicles.

A multifunctional underwater microrobot for mother-son underwater robot system

In past few years, various underwater microrobots have been developed with the micromachining technology, development of smart actuators and biomimetic applications. Most of them possess the attributes of flexibility and compact structure. However, their endurances, moving speeds, and load capacities for power supplies and sensors are limited for the small sizes. So, these microrobots are not suitable for the real-world applications. To solve these problems, we proposed a mother-son robot system, which included several microrobots as sons and a newly designed amphibious spherical robot as the mother. The mother robot was actuated by four water-jet propellers and eight servomotors, capable of providing a stable high speed and carrying the microrobots to a desired target location where tasks were to be performed. To implement the mother-son robot system, we proposed a multifunctional underwater microrobot and developed the prototype by using ionic polymer metal composite (IPMC) actuators. The microrobot could perform walking, rotating, grasping, floating, and swimming motions with the compact structure. Then we evaluated its moving speeds in the water tank. At last, we carried out the releasing experiment to demonstrate the feasibility of the mother-son robot system.

An attitude estimation system for amphibious spherical robots

As one of the most effective tools for exploring the ocean, automatic underwater vehicles have attracted a lot of attentions for years. But some key problems have not been solved properly. It is especially difficult to design underwater vehicles in small size. In this paper, three inertial sensors were adopted to fabricate an attitude estimation system, which provided posture information for our amphibious spherical robot to realize motion control and autonomous navigation. The pitch, roll and heading angel were acquired from current robot attitude matrix, which was calculated from the quaternion algorithm. And the attitude was corrected by the fusion of accelerometer and magnetic sensor. Experimental results verified the validation and precision of the robotic attitude estimation system. It has manifested that the system is effective to realize the robot control and navigation.

An adaptive compressive tracking algorithm for amphibious spherical robots

As a critical important function for autonomous mobile robots, visual tracking is a challenge work in the field of computer vision, for the reason that factors like illumination variance, partial occlusions and target appearance changes shall be carefully considered. Focus on applications of our amphibious spherical robots, an adaptive visual tracking algorithm was proposed on the basis of compressive tracking. A feature selection method was designed to choose random Haar-like feature templates in various scales by calculating Fishers criterion functions of features. On this basis, a random feature pool, which tried to preserve discriminative features at different frames, were constructed and then maintained on-line to provide candidate appearance model of the target. Moreover, an adaptive update mechanism was adopted for selectively updating feature templates and classifier parameters of the improved compressive tracking algorithm, which alleviated the drift problem. Experimental results with various image sequences demonstrated the effectiveness and robustness of the proposed tracking algorithm, which can meet practical application requirements of the amphibious spherical robots.

A system on chip-based real-time tracking system for amphibious spherical robots:

Aiming at vision applications of our amphibious spherical robot, a real-time detection and tracking system adopting Gaussian background model and compressive tracking algorithm was designed and implemented in this article. Considering the narrow load space, the limited power resource and the specialized application scenarios of the robot, a heterogeneous computing architecture combining advanced Reduced Instruction-Set Computer (RISC) machine and field programmable gate array was proposed on the basis of Zynq-7000 system on chip.Under the architecture, main parts of the vision algorithms were implemented as software programs running on the advanced RISC machine-Linux subsystem. And customized image accelerators were deployed on the field programmable gate array subsystem to speed up the time-consuming processes of visual algorithms. Moreover, dynamic reconfiguration was used to switch accelerators online for reducing resource consumption and improving system adaptability. The word length of accelerators was optimized with simulated annealing algorithm to make a compromise between calculation accuracy and resource consumption. Experimental results confirmed the feasibility of the proposed architecture. The single board tracking system was able to provide an image processing rate of up to 89.2 frames per second at the resolution of 320 × 240, which could meet future demands of our robot in biological monitoring and multi-target tracking.

Design and evaluation of quadruped gaits for amphibious spherical robots

Aiming at exploration tasks in complex amphibious environments, quadruped gaits were designed, implemented and evaluated for our amphibious spherical robot to enhance its adaptabilities to various terrains. A simplified locomotion model of the robot was established to analyze the walking process. Then three types of walk gait were implemented on the robotic platform using FPGA, which provided different stability and adjustable motion speeds to adapt various terrains. Furthermore, the attitude of the robot was estimated online using an inertial measurement unit. And the adopted gait was adaptively adjusted with the acquired compensation value, which ensured that the robot was able to walk on a slope no larger than 20 degrees. Evaluation experiments on robotic motion performance indicated that the amphibious spherical robot was capable of moving stably at different speeds in multiple environments, which enhanced its mobility and viability.