Decentralized decision making and navigation strategy for tracking intruders in a cluttered area by a group of mobile robots
i DECENTRALIZED DECISION MAKING AND NAVIGATION STRATEGY FOR TRACKING INTRUDERS IN A CLUTTERED AREA BY A GROUP OF MOBILE ROBOTS by Muhammad Usman Arif 18 May 2020 ii
TABLE OF CONTENTS
Abstract ......................................................................................................................... iv
CHAPTER 1.
Introduction ........................................................................................ 1
Problem Statement ......................................................................................... 2
Research Questions ........................................................................................ 3
Aims and Objectives ...................................................................................... 4
Methodology .................................................................................................. 4
Structure of the Report ................................................................................... 5
CHAPTER 2.
Literature review ................................................................................ 6
Robot Navigation ........................................................................................... 6
Target detection using vision sensor .............................................................. 9
Conclusion ................................................................................................... 10
CHAPTER 3.
Methodology and Algorithms .......................................................... 12
Introduction .................................................................................................. 12
Designing Tools ........................................................................................... 12
Software and Programming Language used ....................................... 12
Robot and Sensors Selection ................................................................. 13
Kinect Sensor for vision and depth measurement .............................. 14
Navigation Path Algorithm .......................................................................... 15
Arithmetic Mean Based Navigation Strategy ...................................... 15
Intruder Detection via Image Processing ..................................................... 18
Intruder Detection by Colour Segmentation ....................................... 19
Probability of Danger Algorithm ................................................................. 20
Assumptions ................................................................................................. 22
Programming and algorithms ....................................................................... 23
Conclusion ................................................................................................... 30 iii CHAPTER 4.
Results .............................................................................................. 31
Simulation Environment .............................................................................. 31
Robot Navigation Through Obstacles .......................................................... 32
Intruders Detection and Intruder Following ................................................ 38
Conclusion ................................................................................................... 44
CHAPTER 5.
Discussion and Conclusion .............................................................. 45
Future Recommendations ............................................................................ 47
References .................................................................................................................... 48 iv ABSTRACT
In the current era of industrial revolution, the mobile robots are playing pivotal role in helping out mankind in many complex and hazardous environments for performing tasks like search and rescue, obstacle avoidance, mining and security surveillance etc. A lot of navigation algorithms has been developed in recent years but novel challenges still exist in autonomous path planning of multiple robots to track and follow multiple intruders. This report demonstrates a decentralized strategy of arithmetic mean based navigation algorithm for a group of mobile robots to navigate through an unknown environment filled with obstacles to detect and follow multiple invading intruders. The suggested navigation strategy ensures that mobile robots safely move right in the middle of surrounding obstacles to maintain safe distance and to avoid collision with obstacles and each other. The conventional method of color recognition is used to detect dynamic intruders and calculate pixel values using Microsoft Kinect sensor camera. A probability of danger algorithm is introduced to ensure that all the intruders present in the environment are being followed by friendly robots on the bases of minimum distance between intruder and its follower. The mobile robots follow intruders movement on the bases of their pixel values. The low pixel value means that intruder is far away and high pixel value represents that intruder is closer to the friendly robots. All the algorithms and image processing technique are implemented and tested in WEBOTS simulation environment using C programming language and the results show the success of proposed arithmetic mean based navigation and probability of danger based intruders following algorithms. 1
CHAPTER 1.
INTRODUCTION
The development of industrial automation applications in manufacturing, military operations, healthcare, oil and gas, agriculture and mining etc. has resulted in use of mobile robots for performing multiple tasks in an industrial environment [1]. The ground based mobile robots can be used in many complex and hazardous environments for performing tasks like search and rescue, obstacle avoidance, path planning, relocating objects, navigation control, intrusion detection, and object trapping etc. The basic problems of autonomous ground robots involve decentralized navigation and detection of targets. So development of an algorithm, for reliable navigation control and path planning for multiple robots while avoiding obstacles present in their path and finding the multiple target intruders, is of great practical importance and a hot topic of discussion among researchers in last decade. The navigation control system of a robot to work autonomously in real time involves three functions which are path planning through obstacles [2], self-localization in suggested environment [3], and motion control to reach the final goal [4]. First of all, the part of path planning involves the decision making of a robot to find an optimal way to avoid all the possible obstacles present in the environment. Secondly, the robot should be able to locate its exact position in the environment which is dependent on the application for which the robot is being used. Thirdly, robot motion is an important task which makes the robot to perform specific duties according to the application for which it is being used. There are many types of applications for which ground robots are being used to search and to detect the intruders in different environments [5]. Due to increase in number of hostilities among state actors and non-state actors has resulted in internal security issues of many countries around the world [6]. These security issues has not only resulted in loss of precious lives of armed forces protecting the border but also putting the lives of common citizens of a country at risk. So the use of ground robots in these sensitive and critical areas around the border for surveillance and detecting intruders will be of great benefit, not only for protecting the precious lives of armed forces but also less human physical involvement will be required. The robots used for 2 these type of applications should be equipped with sensors which enable them to explore the environment to detect the obstacles and intruders present. These surveillance type robots will require enhanced capabilities of navigation and vision to avoid obstacles and detect intruders. A lot of research papers which will be discussed in Chapter 2 of literature review will demonstrate that a lot of work has been done for navigation of ground robots in different environments. Firstly, many researchers such as [7], [8], [9], [10], [11], [12]; focused on developing different algorithms for robot navigation for avoiding obstacles. But in these research papers all the results were shown in 2D simulation environment. So there is still some area of research which is required to show the results in 3D simulation environment. Secondly, most of the researchers such as [7], [8], [10], [12], [13], [14]; focused on navigation of single robot in an environment. So there is still an area of research for developing strategies for navigation of multiple robots in same environment or experimental terrain. Thirdly, many researchers focused on developing strategies for a robot navigation to reach a fixed target position. Although research is also available for finding targets in unknown position but still research in this area is in early stages and more effective strategies can be developed. Lastly, following moving targets or intruders is not an old problem and currently a lot of research and strategies are being proposed such as [13], [15] and [16] etc. But still there is work needs to be done for detecting multiple intruders present in an area and a method which enable friendly robots to trap or follow all the intruders present.
Problem Statement
In recent years, service based robots became popular and started being used for many application such as vacuum cleaners, food delivery and smart wheelchair, which only required the functionality of avoiding obstacles and mostly fed with the final target positions. But search based robots are different than service based robots. Search based robots require some additional functionalities which provides novel challenges like vision capability to detect specific targets and a navigation algorithm which enable them to operate autonomously irrespective of other friendly robots present in the environment. The navigation strategy needs to be developed for robots 3 to pass through obstacles so that they maintain a safe distance from all the obstacles. The vision capability is required to recognize moving hostile targets or intruders present in the area of interest. These intruders can be at any location, so the robots should be able to explore the environment in such a way that they are able to locate the intruders and then an algorithm is required which will enable them to follow the intruder which is closer to them. An image processing technique will be required which will equip the friendly robots to recognize an intruder in their field of view. A strategy is required which will make sure that friendly robots should not collide with each other and maintain a safe distance between themselves too.
Research Questions
Here are some of the challenging research questions which are very important for understanding the purpose of this report • What type of robot should be used which will provide a control system feasible for integrating all the sensors and equipment required for the completion of this report? • What type of algorithm should be developed which will enable all the friendly robots to navigate through obstacles and also enable them to operate autonomously in a decentralized fashion while avoiding collision with each other? • What type of vision technology and image processing technique should be used to identify the intruder in a specific environment and what type of strategy should be implemented which will enable friendly robots to differentiate among different intruders? • What type of algorithm should be devised which will enable a friendly robot to follow a specific intruder robot if multiple intruder robots present in their field of view? • What type of software and simulation environment should be used to show the final results and limitations faced in the report? 4
Aims and Objectives
The need for developing a decentralized navigation system for multiple robots operating in a cluttered area and detecting intruders has resulted in motivation for doing research and devising a strategy to overcome the proposed challenges. So following are the aims and objectives of this research report • To develop a strategy for ‘decentralized multiple robots path planning’ while moving through cluttered environment • To construct a method for detecting intruders whose positions are unknown • To develop a mechanism for trapping or following intruders • To develop a control system for robots while using a camera, range sensor and distance sensors
Methodology
The development of strategies for robot navigation and detection of intruders in an environment filled with obstacles divided into three parts. First of all, an arithmetic mean distance based algorithm is developed for navigation of robot between obstacles. This arithmetic mean based navigation algorithm uses onboard sonar sensors of robot to measure the distance from obstacles on right and left hand side continuously and calculate the average of values between both sides to determine the next points where robot should go and navigate easily between obstacles without colliding. Secondly, the robot will be able to detect the intruders differentiated by different colors in the environment using a camera installed above robot and an image processing algorithm. Thirdly, a range sensor will be used to measure the distance of all the intruders from the robot. The range sensor will also be installed above the robot. A probability of danger algorithm is devised which enable a robot to follow the intruder which is closer than other intruders. The detail of these algorithms and strategies will be discussed in Chapter 3 of this report. The complete algorithm is programmed using C language and same program will be used for all the controllers 5 of the friendly robots. All the algorithm strategies are tested separately and then together as a complete control program.
Structure of the Report
The remainder of this research report discusses the background and literature review in Chapter two which validate the aims and objectives described above. The detailed methodology adopted, algorithms and programming is presented in Chapter three. Chapter four discusses the results obtained via simulation on the software and discussion of obtained results in relation to aims. Chapter five concludes and discusses the finding of this report and presents the future possibilities of research. CHAPTER 2.
LITERATURE REVIEW
In this chapter, a careful literature review of navigation strategies of a robot avoiding obstacles and finding target using vision sensors is done. The three most important parts of this research report are robot navigation through obstacles, detecting an intruder using vision sensor and following or trapping the intruder.
Robot Navigation
There are many algorithms proposed by researchers over the years for navigation of ground robots avoiding obstacles and reaching a target point. Here are some of the important works presented by authors In [7], the author in his PhD dissertation discussed the robot navigation bug algorithm in 2 dimensional environment and introduced tangent bug algorithm for finding shortest path to a final set point. Although robot was operating in an unknown environment for finding shortest path but still final position needed to be set for termination of robot program. Although significant improvement in results were recorded but still there were many shortcomings which needed attention. First of all, bug algorithm performance differs significantly depending on the environment it is being operated. Secondly, all the tests were conducted in 2 dimensional environment. Thirdly, moving around an obstacle region and hope to find a final position may not be fully autonomous system. In [10] and [17], the authors discussed the development of A star (A*) algorithm for collision free path planning for a robot. A star algorithm is very popular technique for determining the minimum distance between two points. This algorithm detects the corners of the obstacles present in robots field of view and make a pre-emptive decision to follow a path which will avoid obstacles. The results showed the increased efficiency in determining the shortest distance with decreased time. But the issue with this type of technique is the size of the environment. If the size of environment increased then execution speed of algorithm will be decreased. Also this 7 algorithm is developed for only one robot operating in an environment and does not discuss group of robots working in the same environment. In [18], [19], [20]; the researchers presented a fuzzy logic controller based algorithm for navigating a robot in the environment. A fixed target point is achieved by moving a robot from a starting point. Infrared sensors installed on the robot which detected the obstacles around the robot and give this information to controller which apply fuzzy logic algorithm and change the speeds of right and left wheels of the robot. This approach does introduce a new algorithm for navigating through not only static obstacles but also dynamic obstacles. Although this algorithm is suitable for dynamic obstacles detection but still it does not address the issue of dynamic target. So this approach is only applicable when targets final position is known. In [9], the author discussed the navigation control of a team of robots moving in unknown environment with obstacles and to trap a target while maintaining a minimum distance among themselves. This work highlights the smart use of energy consumption and complex computations at fixed intervals. All the robots are autonomous in making decision for choosing the best path available. The algorithm developed is focused on reducing the computational time and consumption of energy. Significant results were recorded but in this research work all the robots were fed with information of final target which is static. In [21], the author introduced a new hybrid algorithm of navigation for robot moving in an environment with dynamic obstacles and goals. This algorithm uses a mechanism by detecting obstacles and goals using a camera. The encouraging results were obtained which were better than earlier algorithms proposed for navigation of robot through dynamic obstacles. But still this algorithm will not able perform if multiple robots are employed in same environment for various tasks. In [22], the author discussed the algorithm development for a robot navigation in an unknown environment for finding objects and relocating those objects to a specific place. For navigation and finding an exploration path in an unknown environment bug algorithm was used. Secondly, for object detection, which is required to be relocated, image processing techniques like color segmentation and template matching were used by installing a camera above robot. Thirdly, two types 8 of robots were used to measure the efficiency and performance of different robotic control systems for performing the same task. Again a significant improvement of results in 3 dimensional environment were recorded. But there are still many research issues which could be included for improving performance. First of all, image processing techniques used require complex computation capacity and more processing time. So this has made one of the robot slower. Secondly, as robot is moving along the boundary following bug algorithm and searching for objects and starting point at the same point. If the robot find starting point first instead of target object then robot will consider the search process is fully complete and all other targets are not reachable. Also the targets used for experimentation are considered static instead of dynamic. Thirdly, two robots were designed of different structure and tested separately instead of group of robots operating together in same environment. One robot performed better due to its designing and construction. In [23] and [24], the author introduced a position estimation switching algorithm (PESA) for avoiding convex obstacle and switching navigation algorithm to avoid multi-obstacles in a region by a group of robots in a 2D area. The team leader of robots decides the path to follow in few simple computations and other robots follow the leader. The results does validate the efficacy of proposed algorithms but scope only limited to obstacle detection. In the same way, the authors proposed a distributed control strategy for detection of intruder entering in a protected area by group of robots in [25]. Another paper [26], discussed the probability of intruder detection by a group of robots using a intelligent game theoretic approach. In these two papers of intruder detection, the simulation was done in 2D environment without the deployment of obstacles in the region. Lastly, another algorithm [27], which was formulated to avoid dynamic obstacles with characteristics of changing shapes and sizes. This algorithm considered obstacles as virtual repulsive force and region between obstacles as virtual attractive force to navigate a robot through the region. Again scope is limited to avoid obstacles in the region and simulation is conducted in 2D environment. So there is still an open area of research, in which a fleet of robots should be able to detect and avoid obstacles while keeping track of intruders in an environment. 9
Target detection using vision sensor
The availability of low cost, low power, high resolution and high speed processing cameras has made it very commonly used for vision purpose in ground robots these days [28]. The purpose of camera is to provide data, composed of images of the environment, to the robot. The robot then apply some image processing technique to derive some desired information which is used for detecting an intruder and also useful for making the robot to follow the intruder. Following are some of the vision sensors and techniques being used by researchers In [14], the author designed a robot using a camera which detects the target object by using image segmentation technique and also follow the target object maintaining a specific distance. The technique used is of great importance as it is also being used to great extent in current report. But there are some areas where further research could have been done. Only image processing technique is used in this report and no algorithm proposed in case of any obstacle present in the path of robot. The experimental results shown by this experiments used only one target object. The most relevant research report using a camera above robot for stereo vision and convolutional neural network tracker algorithm to follow a person is presented by [15]. The robot able to detect a moving person using camera in such a way that enable it to calculate the distance from target person. The convolutional neural network algorithm enable the robot to recognize the direction of target person in which it is moving. So even when the target is not in the field of view of robot, it will still be able to follow the target depending on its last known location. This algorithm showed significant results following a moving target. But still there are some areas of research which are ignored in this report which include only one moving target is considered during experimentation and no obstacle avoidance algorithm used. In the following research papers [5],[13], [16], [29], [30]; the authors used Microsoft Kinect sensor to detect target objects. The Kinect sensor has an RGB camera and rangefinder which enables the robot to not only detect the target object but also the distance from the target object. This Kinect sensor is very useful for calculating the distance of robot from multiple and follow them. The drawbacks of these papers include only one target object or static target objects. 10 In [22], the author described many image processing techniques such as image segmentation [31], template matching [32], scale invariant feature transform (SIFT) [33] and speeded up robust features (SURF) [34]. Firstly, the image segmentation is very simple and basic technique which detects different colors and textures in an image. So if intruders present in an environment be given a specific color, then it will be very easy to detect them by using simple image segmentation technique. Secondly, in template matching a robot is fed with the template of object image which needed to be detected. If template of object image matches in image provided by vision sensor then its very easy to detect intruders. The template of object image will always be smaller than actual template of image received from camera. Thirdly, SIFT and SURF are some advanced image processing techniques which can be used to detect even fine features of an object in an image. As high computation is required for template matching, SIFT and SURF image processing techniques, so simple image segmentation is very suitable for this research report.
Conclusion
So researchers have devised many algorithms and techniques for robot navigation avoiding obstacles and detecting and following targets using vision sensors. Almost all the algorithms have their own limitations and better results can be achieved by combining different algorithms and techniques together. First of all, for navigation of robot through obstacles no already proposed algorithm like Bug, A star, Fuzzy logic or new hybrid is used for this report. Instead a simple algorithm based on arithmetic mean values of distances from obstacles is formulated which enables the robot to navigate through obstacles without colliding them. Secondly, intruders are detected by using color segmentation technique as compared to other image processing techniques as this requires less computation and processing time. Thirdly, instead of using camera and distance sensor separately, Kinect sensor is used as it provides both camera for vision and rangefinder to measure the distance from intruders. An algorithm of probability of danger is developed which provide the friendly robots to follow the target which is nearer to them. The detail of all these algorithms and techniques will be described in next Chapter three. 11 The novelty of this research report is going to be the originality of probability of danger and arithmetic mean based algorithms which are specifically not been described in any research paper described above. Also the simplicity of sensors interaction with each other and achieving the goal by using limited number of sensors is also an important aspect of this research report. CHAPTER 3.
METHODOLOGY AND ALGORITHMS
Introduction
An environment or terrain, in which a robot move, usually have a starting point from where the robot will start its movement, a final target or goal which the robot has to achieve and multiple obstacles of different sizes and shapes. The aim of robot is to move continuously for finding an optimal path that enables it to achieve final goal while avoiding obstacles in its path. The system which enables the robot to achieve all these tasks is known as navigation system. The navigation system of robot uses the essential data or information of the environment which it gathers by using onboard sensors or devices for deciding its movement. As the final target or intruder position is unknown so robot should plan its movement in such a way that enables it to detect target at any position of the environment [22]. The environment considered for this research report is an indoor environment that means it is surrounded by boundary walls and robot will have to perform all the tasks within these walls. In this chapter all the designing tools and software use for this project are discussed. The algorithm which is being used for navigation of robot in designed environment is elaborated in detail. The use of image sensor and image segmentation technique, which are used to detect the intruders, are discussed. After that the probability of danger algorithm is discussed which makes the robot intelligent enough to follow the nearest intruder. All the limitations and assumptions which are considered in designing the environment and execution of program in this report are also discussed separately. In the last part, the programming or code which is used is explained.
Designing Tools
Software and Programming Language used
13 First of all, there many software and programming tools which are being used to program robots by researchers like ROS (robot operating system), MATLAB, CYBERBOTICS and GAZEBOISM etc. all over the world. The software which is used for programming and designing of simulation environment for this report is WEBOTS [35]. WEBOTS is an open source robot simulator. This simulator is commonly being used around the world in different universities for research purpose. The advantages of using this software are that it provides the ability to design 3D environment which is just same as a robot operating in real environment and it also supports multiple programming language such as C, C++, Python, MATLAB and java etc. Due to advantage of using multiple programming language in a single software make it easy to do complex tasks like image processing. All the robots controllers are programmed using basic C language. Basically a single program algorithm is developed which is being operated on all the controllers of the robots. The are some built-in libraries in WEBOTS software and some basic libraries of C language which are used for programming of robots.
Robot and Sensors Selection
The robot selected for programming and simulation for this report is Pioneer 3-DX which is part of Pioneer family of robots [36]. This robot device is being used around the world by many advanced research institutions. As designing a robot is not a part of this project, so selecting Pioneer 3-DX as a test robot has provided all the necessary onboard tools and sensors which is enough for execution of all the tasks intended as shown in Figure 1. The Pioneer 3-DX has 16 onboard ultrasonic (sonar) sensors from which are 8 sensors are on front side and 8 on rear side. All the sensors are equally spaced covering 20 degrees of angle between each other except for four side sensors which have angle of 40 degrees as shown in Figure 2 [37].
Figure 1: Pioneer 3-DX with Kinect Sensor
14 For this particular report only front side eight sonar sensors will be used to measure the distance from obstacles as there is no need to measure the distance on rear side of the robots. Sonar sensors operate in different way than conventional distance measuring sensors, which means if reflected rays from obstacles are greater than the angle of 45 degrees of reflection cone of aperture never return the distance value as explained in Figure 3 [38]. The sonar sensors can detect an obstacle within 5 meters distance.
Figure 3: Reflection of sonar ray
Kinect Sensor for vision and depth measurement
Now for detection of intruders in the simulation environment, a vision sensor (camera) is required. Apart from vision, a distance sensor or range finder is also required which will measure the distance of robot from the intruders. Both of these functionality can be achieved by connecting a simple Microsoft Kinect sensor with a robot as shown in Figure 1. The Kinect sensor, which is shown in Figure 4 [39], is easily available in market and has been used by many researchers in their projects as already explained in chapter two.
Figure 2: Sonar sensors position Figure 4: Kinect sensor with camera and range finder capability 3.3
Navigation Path Algorithm
There are many algorithms for path planning of a robot avoiding obstacles such as A star, Bug and fuzzy logic controller algorithms etc. which has been discussed in chapter two. The most popular among these algorithms is Bug algorithm in which a robot follow the wall of obstacle till it reaches the final goal position. But in this report, any algorithms discussed above, has not been used. But a new strategy of measuring distances from obstacles on left and right hand side of a robot and taking their arithmetic mean value to decide the navigation of robot in the simulation environment is used.
Arithmetic Mean Based Navigation Strategy
As ultrasonic (sonar) sensors are installed on Pioneer robot, they will measure the distance from obstacles and navigate the robot through obstacles. So a force or velocity will be required to move the robot wheels. Hence
𝑉 = {𝑣 (𝑡) , 𝑣 (𝑡) , … … , 𝑣 𝑛 (𝑡) } (1) 𝑊 = {𝑤 (𝑡) , 𝑤 (𝑡) , … … , 𝑤 𝑛 (𝑡) } (2) Where 𝑉 is the linear velocity of a robot and 𝑊 is the angular velocity of the robot at any time t and n is the number of robots present in the simulation environment. Now at any instant during simulation the linear and angular velocity should always satisfy the following condition 𝑉𝑚𝑖𝑛 < 𝑣 (𝑡) = 𝑣 (𝑡) = ⋯ = 𝑣 𝑛 (𝑡) < 𝑉𝑚𝑎𝑥 (3) 16 𝑊𝑚𝑖𝑛 < 𝑤 (𝑡) = 𝑤 (𝑡) = ⋯ = 𝑤 𝑛 (𝑡) < 𝑊𝑚𝑎𝑥 (4) Now let’s consider So (n) be the ultrasonic (sonar) sensors installed on Pioneer 3-DX robot where n is the total number of sonar sensors. As there are total 8 sensors installed on robot so n = (0,1,2,3,….,7). Now four sensors will be present on left front side and four sensors on right front side of the robot. So let’s consider The left front side sensors of the robot will be donated as 𝑆𝑜 𝑚𝐿 where ∀ 𝑚𝜖𝑛 , 𝑚 = 0,1,2,3 (5) So all the sensors on left side can be denoted in an array 𝑆𝑜 𝑚𝐿 = [𝑆𝑜 , 𝑆𝑜 , 𝑆𝑜 , 𝑆𝑜 ] (6) The right front side sensors of the robot will be donated as 𝑆𝑜 𝑘𝑅 where ∀ 𝑘𝜖𝑛 , 𝑘 = 4,5,6,7 (7) So all the sensors on right side can also be represented in an array 𝑆𝑜 𝑘𝑅 = [𝑆𝑜 , 𝑆𝑜 , 𝑆𝑜 , 𝑆𝑜 ] (8) We suppose that a robot, represented as R , is passing through an environment filled with obstacles. Now when a robot navigate between left and right side obstacles then it will detect the distance from them using sonar sensors in such a way as shown in Figure 5. As it has already been mentioned that only those sonar sensors rays will be detectable which are within their detectable range. So 17 Figure 5: Robot movement between obstacles
In the above scenario, two sensors on left side has detected the obstacle and two sensors on right side has detected the obstacle. Then equation (6) and (8) will become like 𝑆𝑜 𝑚𝐿 = [𝑆𝑜 , 𝑆𝑜 , 0, 0] and 𝑆𝑜 𝑘𝑅 = [0, 0, 𝑆𝑜 , 𝑆𝑜 ] respectively. So all the values of sensors which are zero will be ignored and rest of the values will be saved. As a first step of arithmetic mean based algorithm let’s use the values of left side sonar sensors of robot 𝑆𝑜 𝑚𝐿 ̅̅̅̅̅̅ = ∑ 𝑆𝑜 𝑃𝐿 𝑇𝑜𝑡𝑎𝑙 𝑛𝑜. 𝑜𝑓 𝑛𝑜𝑛𝑧𝑒𝑟𝑜 𝑣𝑎𝑙𝑢𝑒𝑠⁄ , 𝑤ℎ𝑒𝑟𝑒 𝑆𝑜 𝑃𝐿 > 0 (9) If same mechanism is applied to right side values then 𝑆𝑜 𝑘𝑅 ̅̅̅̅̅̅ = ∑ 𝑆𝑜 𝑃𝑅 𝑇𝑜𝑡𝑎𝑙 𝑛𝑜. 𝑜𝑓 𝑛𝑜𝑛𝑧𝑒𝑟𝑜 𝑣𝑎𝑙𝑢𝑒𝑠⁄ , 𝑤ℎ𝑒𝑟𝑒 𝑆𝑜 𝑘𝐿 > 0 (10) Now after calculating the mean values of distances on each side of robot let’s calculate the arithmetic mean value of these obtained values. 𝑃̅ = 𝑆𝑜 𝑚𝐿 ̅̅̅̅̅̅ + 𝑆𝑜 𝑘𝑅 ̅̅̅̅̅ 2⁄ (11) Hence, 𝑃̅ is the point where the robot needs to go next as shown in Figure 5. So theoretically, 𝑃̅ will be point which is approximately in the middle of obstacles. 𝑆𝑒𝑛𝑠𝑜𝑟𝑠 𝑆𝑜 𝐿 𝑎𝑛𝑑 𝑆𝑜 𝐿 Left side obstacle R 𝑃̅ Right side obstacle
𝑆𝑒𝑛𝑠𝑜𝑟𝑠 𝑆𝑜 𝑅 𝑎𝑛𝑑 𝑆𝑜 𝑅
18 Now to navigate the robot to move either towards left or right side following conditions are applied • If 𝑃̅ < 𝑆𝑜 𝑚𝐿 ̅̅̅̅̅̅ then robot right wheel velocity will decrease and left wheel velocity will remain same. This will make the robot to move towards right side until following condition is not met −0.015 < 𝑃̅ − 𝑆𝑜 𝑚𝐿 ̅̅̅̅̅̅ < 0.015 (12) If following condition satisfied then velocity of both wheels will become same again and robot will try to stay within threshold limit as defined in above equation. This minimum threshold limit is defined after experimenting different values of threshold in the simulation environment. • If 𝑃̅ > 𝑆𝑜 𝑚𝐿 ̅̅̅̅̅̅ then robot left wheel velocity will increase and right wheel velocity will remain same. This will make the robot to move towards left side until following condition is not met − ̅ − 𝑆𝑜 𝑘𝑅 ̅̅̅̅̅̅ < 0.015 (13) If following condition is also satisfied then again robot will try to stay within threshold limit. So the above algorithm or strategy for robot navigation does ensure that robot safely pass through area filled with obstacles without any danger of collision as robot sensors are continuously measuring the distance from obstacles and updating the next point where robot should go. Furthermore, the strength and weaknesses of this proposed strategy will be discussed in Chapter 5. Intruder Detection via Image Processing
Now after developing a mechanism for robots navigation system to avoid obstacles, the second task is to detect intruders in the simulation environment. As these intruders are dynamic and can change their location, so a vision sensor is going to be required. For that the camera of Kinect sensor is going to be used as already mentioned earlier in the designing tools section. To detect intruders in the simulation environment, image processing technique will be required. There are many image processing techniques like template matching, color segmentation and speeded up 19 robust features etc. But for this project color segmentation technique is applied for detecting intruders.
Intruder Detection by Colour Segmentation
Color segmentation is one of the simple and basic image processing technique which is being used by researchers to detect different colored objects in an image. Basically all the colors are made by mixing the different concentrations of red, green and blue known which are also known as RGB. As an image is comprised of pixels and each pixel can be processed in such a way that intensity of each color of RGB can be measured. The intensity value for each color varies between 0 to 255 and can be saved in 1 byte (8 bits). So if intensity of R is 255, G is 0 and B is 0 in a pixel of image then that image is of red color. As values of RGB is being calculated for each pixel of a image, so returned data is of 24 bits. As an example if color segmentation is applied on an image for detecting red color as shown in Figure 6 which is taken from [22] in such a way that detected red pixels should be represented as white pixels and all other pixels should be black then as a result we get image as shown in Figure 7 [22].
Figure 6: Camera image of can Figure 7: Color segmented image
In this project report, two intruder robots needs to be detected as shown in Figure 8. Each intruder robot is assigned by a different color so that searching robot can differentiate between them. When image processing technique is applied to detect them via camera, the result is shown in Figure 9. 20
Figure 8: Intruders Figure 9 :Color segmented image
As successful results for detecting are achieved by just applying simple image processing technique so there is no need to opt for complex image processing techniques. As this software simulation environment does behave same like real world environment, so even if light intensity changes in the environment the robot will still be able to detect the intruders based on their colors.
Probability of Danger Algorithm
After successful navigation between obstacles and detection of intruders by a robot, the next task is to either follow the intruder if it is moving or trap it if it is stationary. The friendly robots should maintain a specific distance from the hostile intruders. The friendly robots should also be able to avoid collision with each other. After assigning colors to intruders, it is very easy for friendly robots to ignore all other robots in the environment and consider them as obstacles. So when a friendly robot come in range of another robot, then same algorithm of arithmetic mean to avoid obstacles will apply. To follow only one intruder is very easy for a group of robots when they detect it by using camera. On the other hand, there are two intruders being deployed in the simulation environment for this project. So friendly group of robots has to make a decision to follow intruders in such a way that each intruder is being followed by at least one robot. So, to achieve this task a probability of danger algorithm is devised. 21 Generally, consider if there are two hostile targets or intruders which are approaching towards a person then the target which is closer to the person is more dangerous than other. Same technique is also applied to make a decision by robot to follow an intruder in this report. So the robot should follow the intruder which is closer to it, if multiple intruders are in its field of view. The friendly robots first detect that which target is nearer than other, then robots should be able to calculate distance from each of them. Hence, a range finder is used which comes as a function or part of Kinect sensor. By using range finder or also known as depth finder robot will calculate the distance from intruders in its field of view as shown in Figure 10
Figure 10: Distance between intruders and robot
In above figure, robot should be able to make the decision to follow green robot as it is closer than the blue robot. This algorithm also works in situations when a friendly robot is following an intruder and suddenly another intruder comes in field of view, which is closer than the intruder which robot is following, the friendly robot will change the target and start following the robot which is closer now. This probability of danger technique for following multiple robots in a simulation environment is very simple and important but novel in its own way. This technique make this project unique and special in this regard. 22
Assumptions
There are multiple assumptions which are made during the designing of simulation environment for this report and they are as follows • All the sensors used in this project will return perfect values which is not the case in real world environment where sensors are prone to return false or incorrect values due to noise. • There is no separate program written for intruders to move freely in the simulation environment. But these intruders can easily be moved manually with the help of keyboard and mouse in the software. But during the simulation in this project, readily available code in the libraries of WEBOTS software for moving intruder in a straight line is used. • A minimum distance is considered between the obstacles so that robot can easily navigate between obstacles. • All the mechanical parts and sensors used in this project will work perfectly and no mechanical issue will be faced as all the work is done in simulation environment. • There are four friendly robots used for navigation between obstacles and intruder detection. Two intruder robots are used which are assigned different colors so that they can be detected easily using camera. • For applying probability of danger algorithm, the simulation environment is designed in such a way that both intruders will be in the field of view of friendly robots so that each one of them can make a decision by using danger algorithm. 23
Programming and algorithms
Pseudo-code or algorithm which is followed in implementation of all the proposed techniques is shown in below pseudo-code Figure 11 1.
Initialize all robots, sensors and devices 2.
Check for Obstacles in surrounding area 3.
If (obstacles detected) then Measure distance from obstacles Apply arithmetic mean based navigation algorithm 4.
Else If (no obstacle detected) then Find intruders using camera Apply color segmentation technique If (multiple intruders detected) then Calculate distance from intruders using range finder Apply probability of danger algorithm Follow the intruder nearer to the robot Else if ( single intruder detected) Calculate distance from intruder Follow intruder If (detected pixels of intruder which robot is following > 3000) then Robot is in stationary position Else Keep following If (obstacle is detected again) Go to Step 3
Figure 11: Pseudo-code of robot navigation and intruder detection
24 The algorithm which is used to detect and follow intruder is given in Figure 12 1.
Divide camera image in 2 parts (Left half of image & Right half of image) 2.
Do image processing on left half of image (color segmentation) A.
Detect and count blue intruder pixels in left half B.
Detect and count green intruder pixels in left half 3.
Do image processing on right half of image A.
Detect and count blue intruder pixels in right half B.
Detect and count green intruder pixels in right half 4.
Calculate distance from each robot A.
If (range from blue intruder < range from green intruder) Add blue robot total pixels and ignore green intruder B.
Else if(range from blue robot > range from blue robot) Add green robot pixels and ignore blue robot 5.
For following intruder left and right A.
If (pixels on right half < pixels on left half) Move robot to left B.
Else if (pixels on right half > pixels on left half) Move robot to right C.
Else if (pixels on right half ==pixels on left) Go straight 6.
If (total detected pixels < 10) Keep searching Intruder Else if ( total detected pixels > 3000) Stop Robot
Figure 12: Intruder detection and following algorithm
25 The flowchart of above strategy is shown in below Figure 13
Figure 13: Flowchart of robot navigation and intruder detection
As already mentioned that basic C language was used for the implementation of all the algorithms in WEBOTS software. In below Figure 14 all the libraries and basic functions used for programming for this project are displayed.
Yes No Yes No Yes No Yes
Measure distance from Obstacles Search Intruders via Colour Segmentation Search point where robot needs to go (Arithmetic Mean Algorithm) Any Obstacle still being Detected Initialize Calculate the distance from each intruder Follow the intruder which is closer to the robot (probability of Danger algorithm) and calculate pixels Check for Obstacles
Any intruders Found
If detected Pixels>3000 Stop (no motion) Figure 14: Libraries used for programming
After defining the libraries used in this project, below Figure 15 is displaying function in which all the robots, devices and sensors is initialized
Figure 15: All devices and sensors initialized
Now in main function first of all the distance (sonar) sensor will measure whether there is an obstacle is around or not. If there is an obstacle then arithmetic 27 mean based navigation strategy will be implemented as code for that is shown in below Figures 16 and 17 from code line 73 to line 140.
Figure 16: Algorithm for navigation (1)
Figure 17: Algorithm for navigation control (2)
Now when there is no obstacle around the robot then it will try to find the intruder by using color segmentation technique. The robot is programmed in such a 28 way that when it detects an intruder to follow after applying probability of danger algorithm, the robot will try to position itself according to strategy shown in Figure 12 (where algorithm is described in detail), that selected intruder will always be present in the middle of field of view of robot as code shown below Figures 18, 19 and 20 from line 164 to 249.
Figure 18: Intruder detection and following code (1) Figure 19: Intruder detection and following code (2)
Figure 20: Intruder detection and following code (3)
30 Hence the programming strategy which is displayed above has been implemented during the achievement of desired results for this project.
Conclusion
In this chapter, it has been explained in detail the tools and programs which are used for this report project. The algorithm has also been explained in detail. All the strategies and algorithms applied are not of complex nature but unique in their own implementation, particularly probability of danger algorithm and arithmetic mean based strategies of navigation which are the reasons of novelty of this report work. 31
CHAPTER 4.
RESULTS
In this chapter, results are going to be derived and discussed in detail after implementing the methodology explained in Chapter 3 in WEBOTS software.
Simulation Environment
The simulation environment for obtaining the results for this report is designed in such a way that four friendly robots will move through passage having obstacles on both sides. There are multiple obstacles which are dispersed in the simulation environment that can be viewed in below Figure 21. All the friendly robots are clearly marked as Robot 1, Robot 2, Robot 3 and Robot 4 respectively and intruders can also be viewed named as Intruder 1 and Intruder 2 in below Figure 21.
Figure 21: Design of simulation environment
32 The simulation environment is designed intentionally in this way, so that when friendly robots come out of obstacle filled area, they can detect both intruders in their camera field of view which will enable them to apply probability of danger algorithm to decide which intruder they need to follow. In WEBOTS software the required data or results can be viewed in the console which is located at the bottom of software window and can be viewed in below Figure 22.
Figure 22: Data extraction from software
There are different type of results which needs to be displayed to support the hypothesis, aims and objectives of this project.
Robot Navigation Through Obstacles
First of all, the first objective of this report is the navigation of robot through obstacles which is based on arithmetic mean strategy to avoid obstacles. As already discussed above in Chapter 3 that robot needs to stay approximately in the middle of obstacles during motion so that it can effectively avoid collision. 33 After implementation of navigation algorithm and running the simulation for 40 seconds, following data is recorded which is shown below in Table 1. All the values are in meters and taken at 12 different intervals of simulation time while robots are passing through obstacles. During this time of simulation the robots are always detecting the obstacles around them as there is no intruder in sight of these robots. There is also one more condition, that when a robot is detecting obstacle only on either left or right side then the side which is not detecting any obstacle within its range of 5 meters will return maximum value of 5 meters. Then arithmetic mean will be taken by considering 5 meter distance on that side. 34
Table 1: Data representing distances of robots from obstacle
Time in seconds (s)
3 5 8 12 15 19 22 26 29 33 36 40 R o b o t D a t a Distance from left side obstacle (m)
Distance from right side obstacle (m)
Total distance between obstacles (m)
Suggested Position of robot by arithmetic mean algorithm (m) R o b o t D a t a Distance from left side obstacle(m)
Distance from right side obstacle (m)
Total distance between obstacles (m)
Suggested Position of robot by arithmetic mean algorithm (m) R o b o t D a t a Distance from left side obstacle (m)
Distance from right side obstacle(m)
Total distance between obstacle (m)
Suggested Position of robot by arithmetic mean algorithm (m) R o b o t Distance from left side obstacle (m)
Distance from right side obstacle (m)
Total distance between obstacles (m)
Suggested Position of robot by arithmetic mean algorithm (m)
1 0.98 0.96 1.51 1.27 1.08 1.03 1.11 2.06 1.01 1.05 1.05
35 Now next step is to make a meaning of this data and represent results in the form of graphs which will represent the effectiveness of arithmetic mean based algorithm for obstacle avoidance. First of all, let’s consider the data for Robot 1 which is detecting obstacles around it. Now when a robot is measuring the distance from left side obstacles and right side obstacles, then it means that total distance between obstacles is the sum of distances on left and right side obstacle measured by the robot. Now let’s consider distance of robot from right side obstacle is 𝐷 𝑛𝑅 and distance of robot from left side obstacle is 𝐷 𝑛𝐿 ,then total distance between obstacles can be given by 𝐷 𝑛𝑇 𝐷 𝑛𝑇 = 𝐷 𝑛𝑅 + 𝐷 𝑛𝐿 (14) Where n is the robot which is estimating the total distance between obstacles. Now by keeping in view the total distance between obstacles, the navigation algorithm returns a value which will be approximately in the middle of these obstacles where the robot needs to go next. Now the question to ponder, what is the actual location of robot between the obstacles during navigation?. So exact location of robot between the obstacles is the maximum of distances value of left side obstacle and right side obstacle. Now if distance of robot from left obstacle distance is 𝐷 𝑛𝐿 and distance from right obstacle is 𝐷 𝑛𝑅 then the robot actual location between obstacles 𝑃 𝑛 can be represented as 𝑃 𝑛 = 𝑆𝑢𝑝𝑟𝑒𝑚𝑢𝑚(𝐷 𝑛𝐿 , 𝐷 𝑛𝑅 ) (15) Where n is the robot which is navigating through obstacles. This logic will only apply to the values when obstacle is detected on both sides of the robot. So after using the data from above figure, the path which is being followed by Robot 1 is displayed in below Figure 23 36 Figure 23: Robot 1 navigation path
In the above graph, total distance is of robot 1 is calculated using equation 14 and represented with blue line, while suggested path is calculated using equation 11 and represented with red line and actual path followed by robot is calculated using equation 15 and represented with green line. It can be seen clearly that robot tried to follow the suggested path with minor deviation which is due to slow turn rate of robot due to physical attributes within the simulation environment. If a robot is going in one direction and suddenly an obstacle encountered in the path, then robot will take some time to turn towards new path. In the same way of designing the Robot 1 navigation path, the navigation path followed by Robot 2 is shown in below Figure 24. Same equations are used to calculate the parameters which are also represented in Table 1 D i s t a n c e ( m ) Total number of sample values
Robot 1 Navigation Path
Total Distance Between ObstaclesSuggested Position of robot by arithmetic mean algorithmActual path followed by robot Figure 24: Robot 2 navigation path
Again the deviation from suggested path is due to time taken by robot to turn and settle down to new values. But efficacy of suggested algorithm can be viewed clearly that in any instance of time robot was never close to the detected obstacle. Now the mechanism used for robot 1 and robot 2 will also be applied to represent the results for robot 3 path navigation in Figure 25.
Figure 25: Robot 3 navigation path
Robot 3 has actually maintained the best navigation path being followed. This is due to distance between obstacles remained almost constant where robot 3 is navigating. D i s t a n c e ( m ) Total number of sample values
Robot 2 Navigation Path
Total Distance Between ObstaclesSuggested Position of robot by arithmetic mean algorithmActual path followed by robot0123456 1 2 3 4 5 6 7 8 9 10 11 12 D i s t a n c e ( m ) Total number of sample values
Robot 3 Navigation Path
Total Distance Between ObstaclesSuggested Position of robot by arithmetic mean algorithmActual path followed by robot
38 For robot 4 same strategy and equations are used and results are represented in Figure 26.
Figure 26: Robot 4 navigation path
Again robot 4 has tried to position itself right in the middle of obstacles according to the data collected in Table 1. Hence, it can be seen clearly from these graphs that arithmetic mean algorithm for navigation of robot is efficient enough to avoid obstacles and keeping the robot right in the middle of obstacles.
Intruders Detection and Intruder Following
For intruder detection, a camera is being used which is part of Kinect sensor which installed above robots as mentioned in Chapter 3. Two types of colors blue and green are assigned to two intruders which are deployed in simulation environment as shown earlier in Figure 21. After intruders are in field of view of camera then robot detects them by using color segmentation technique. As in Figure 27, a color segmented image is shown for Robot 2 in which robot detected blue and green colors. Also shown is the camera view of each robot detecting both intruders. D i s t a n c e ( m ) Total number of sample values
Robot 4 Navigation Path
Total Distance Between ObstaclesSuggested Position of robot by arithmetic mean algorithmActual path followed by robot Figure 27: Color segmented image for Robot 2
Now as robots are able to detect these intruders, they also measure the distance from each intruder by using range finder sensor. The strategy is very simple that robot measures the distance of only those pixel which are green or blue and this way robot able to decide which robot need to follow. During programming the maximum range which can be measured using range finder is set to 10 meters. Now let’s consider the scenario shown in Figure 27. The measured distance values by each robot to both intruders are shown in Table 2. Now in this particular case Robot 1 follows blue intruder while Robot 2, Robot 3 and Robot 4 follows green intruder.
Colour segmented image after image processing Table 2: Distance of robots from intruders
Robots in Simulation Distance from blue intruder (m) Distance from green intruder (m) Robot 1 6.05 6.55 Robot 2 6.06 5.37 Robot 3 6.28 5.27 Robot 4 8.52 5.51 41 Now let’s consider another case in which intruders position is changed as shown in Figure 28 and measure the distance values again which is mentioned in Table 3. The grey window is actually field of view of range finder which helps it to measure distances. So, Robot 1 and Robot 2 will follow green robot as it is closer to them and on the other hand Robot 3 and Robot 4 will follow blue robot.
Figure 28: Simulation environment in WEBOTS software Table 3: Robots distances from intruders
Now in below Figure 29 shows how robots come near the intruder and stop maintaining distance. Now when intruders are moved manually the robot follows them. When an object is further away from camera then it will appear smaller on Robots in Simulation Distance from blue intruder (m) Distance from green intruder (m) Robot 1 6.35 5.05 Robot 2 5.09 5.05 Robot 3 4.99 5.44 Robot 3 7.29 5.25 42 camera field of view and on the hand when object comes closer than its size increases on camera field of view. So object size is actually depends on the number of pixels occupied in the camera field of view. So logic which is used to maintain a distance from intruder is dependent on number of pixels occupied by the intruder. If number of occupied pixels is greater than 3000 then robot will stop. But as the intruder is moving and number of pixels never increases so the robot also keep following intruder.
Figure 29: Robots following intruders
Now let’s move green intruder robot and check the how friendly Robot 3 and Robot 4 follows the intruder robot. In simulation, green intruder moved in a straight line for 13 seconds and total distance covered by intruder is recorded as shown in Figure 30. Also at four different intervals of time, the distance of Robot 3 and Robot 4 is noted from green intruder as shown in Table 4. 43
Figure 30: Robot 3 and Robot 4 following green intruder Table 4: Robot following an intruder
Time in sec (s) Distance moved by Intruder robot (m) Distance between Intruder and Robot 3 (m) Distance between Intruder and Robot 4 (m) 1 0 1.8 1.82 5 0.65 1.83 1.84 9 1.05 1.8 1.79 13 1.45 1.81 1.8
If these results are shown in graphical form then they can be seen in Figure 31. 44
Figure 31: Graph for Robot 3 and Robot 4 following Green intruder
Now it can easily be deduced from above graph that even when intruder is moving from its initial position and covering distance as displayed by blue line in graph, the distance between intruder and both Robot 3 and Robot 4 remains almost same as represented by green and red straight lines. So those robots are following the intruder wherever it is moving by maintaining a minimum distance to avoid collision. The same strategy can be applied on other two robots and blue intruder and same results will be received. Hence, the aim of friendly robots following intruders in the environment is successfully achieved.
Conclusion
In this chapter, designing of simulation environment is discussed in detail. The methodology which is explained in Chapter three is used to perform all the simulation. Method of obtaining all the data from WEBOTS software simulation and using the data to show the results in the form of figures and tables is also explained. In the next Chapter six, the discussion of this obtained data in the light of aims and objectives of this report will be presented. D i s t a n c e Time
Intruder Following
Distance moved by Intruder robotDistance between Intruder and Robot 3Distance between Intruder and Robot 4 CHAPTER 5.
DISCUSSION AND CONCLUSION
The results obtained above clearly shows the effectiveness of proposed strategies for this report project. There are multiple aims and objectives mentioned in Chapter one which are in consideration while getting these results. Here is the comparison of obtained results and objectives for this project • The first objective is to develop a strategy for decentralized movement of multiple robots in a cluttered environment while avoiding obstacle. An arithmetic mean based navigation algorithm for robots to move between obstacles is developed and target of moving the robots in the middle of obstacles is successfully achieved. This navigation strategy is very novel in its implementation as compare to the navigation strategies of bug algorithm used by [7] and [22], of A* algorithm used by [10] and [17], of fuzzy logic controller algorithm used by [18], [19] and [20] and of hybrid algorithm used by [21].The results which are represented in Graph 1, Graph 2, Graph 3 and Graph 4 do authenticate that robots tend to stay in the middle of obstacles during their movement. The slight error between suggested path and path taken by robot is due to continuous change of distance between obstacles and robot needs to little time to settle on continuous changing path. • The second objective is to construct a method for detecting intruders whose positions are unknown. For getting the desired results, two intruders having different colors are used. The simple image processing color segmentation technique using camera is implemented to detect intruders just like [14], [15] and [22]. The result shown in Figure 17 does indicate that all robots are able to detect intruders with proposed strategy. Also using color segmentation technique and showing these results in a separate display window in WEBOTS software is also successfully implemented, it helps in showing the desired results clearly to mention them in report report. Any advanced image processing technique like SIFT [33], SURF [34] or template matching [32] which can increase robot 46 processing time is not used as aim is to just detect intruder robots and that is achieved by simple color segmentation technique. • The third aim is to measure the distance between intruders and friendly robots and develop a mechanism to follow intruders. A range finder just like [5],[13], [16], [29] and [30] is used for this purpose and results are shown in Table 2 and Table 3. The distance is measured continuously when intruder is in sight of any friendly robot. Then intruder which is closer to the robot is selected as a primary target to be followed by using probability of danger algorithm. This aim is also been successfully achieved and results are displayed in Table 4. A graphical interpretation of results is also presented in Graph 5. This probability of danger algorithm for following multiple intruders is also novel in its implementation. The additional ability of a robot to change primary target when following an intruder and another intruder detected which is more closer than before is not part of additional hypothesis. But this additional functionality has also been achieved during simulation. • In the last, successful integration of Kinect sensor’s camera and range finder for intruder detection and following, while using sonar sensors for robot navigation is successfully implemented. There are also some research questions defined in Chapter one which were focal point while finding results of this report. Here is the short description those challenges in light of results • So Pioneer 3-DX is selected as a test robot which provides all the functionalities and control system for integrating all the sensors and devices together. This robot is readily available in software environment. All the results are successfully achieved by using this robot. • For navigation of robot through obstacles, an arithmetic mean based strategy is successfully designed and implemented. And effectiveness of this strategy is proved via results. 47 • A camera is used for detection of different color intruder robots and results proved its efficacy. A probability of danger algorithm is designed and successfully implemented to follow intruders. This navigation strategy works perfectly when robot is moving through an area having obstacles on both side of robot but issues may arise but not confirmed if an area having obstacles at different places of environment. The robot is going to avoid obstacles in this case too but detection of intruders strategy would be affected if intruder is not in sight of friendly robot. Only intruders present in friendly robot field of view are considered at specific time and probability of danger algorithm is only apply to those intruders to make the robot to follow them follow them. The simulation environment is designed in such a way that all robots detect all intruders and make decision to follow them. Otherwise in different scenario if there is only one intruder in field of view of all robots then only that intruder will be followed. Hence, the research questions, aims and objectives which are highlighted in this report are successfully explained and proved via results. But there are still a lot of improvement which can be achieved as explained in future recommendations.
Future Recommendations
For future work, performance of different algorithms with the performance of purposed algorithm can be compared to evaluate the best navigation strategy through obstacles. In this report, color segmentation technique is implemented but in future different image processing techniques can be integrated to this project to increase the efficiency of detecting objects even during the night time. Instead of two intruders more intruders can be introduced and a mechanism of one robot following one intruder randomly will also be step forward in this report. All the obstacles were considered static during the simulation of this project, so avoiding dynamic obstacles with dynamic intruders will have a great practical and industrial impact on the commercialization of ground robots. 48
REFERENCES [1] A. Khusheef, G. Kothapalli, and M. T. Rad, ‘Simulation of a mobile robot navigation system’, , p. 7, Dec. 2011. [2] F. Colas, S. Mahesh, F. Pomerleau, Ming Liu, and R. Siegwart, ‘3D path planning and execution for search and rescue ground robots’, in , Tokyo, Nov. 2013, pp. 722–727, doi: 10.1109/IROS.2013.6696431. [3] J.-Y. Lee and W. Yu, ‘Robust self-localization of ground vehicles using artificial landmark’, in , Kuala Lumpur, Malaysia, Nov. 2014, pp. 303–307, doi: 10.1109/URAI.2014.7057439. [4] A. Franchi, ‘Human-Collaborative Schemes in the Motion Control of Single and Multiple Mobile RobotsMobile robot’, in
Trends in Control and Decision-Making for Human–Robot Collaboration Systems , Y. Wang and F. Zhang, Eds. Cham: Springer International Publishing, 2017, pp. 301–324. [5] D. S. O. Correa, D. F. Sciotti, M. G. Prado, D. O. Sales, D. F. Wolf, and F. S. Osorio, ‘Mobile Robots Navigation in Indoor Environments Using Kinect Sensor’, in , Sao Paulo, Campinas, Brazil, May 2012, pp. 36–41, doi: 10.1109/CBSEC.2012.18. [6] D. Arjun, P. K. Indukala, and K. A. Unnikrishna Menon, ‘PANCHENDRIYA: A Multi-sensing framework through Wireless Sensor Networks for Advanced Border Surveillance and Human Intruder Detection’, in , Coimbatore, India, Jul. 2019, pp. 295–298, doi: 10.1109/ICCES45898.2019.9002161. [7] J. Ng, ‘An Analysis of Mobile Robot Navigation Algorithms in Unknown Environments’, pp. 1–203, 2013. [8] N. Larasati, T. Dewi, and Y. Oktarina, ‘Object Following Design for a Mobile Robot using Neural Network’,
Comput. Eng. Appl. J. , vol. 6, no. 1, pp. 5–14, Mar. 2017, doi: 10.18495/comengapp.v6i1.189. [9] A. Marzoughi, ‘Decentralised Navigation Control of a Multi-Robot Team to Minimising Energy Consumption in an Unknown Obstacle-Ridden Area’, in , Wuhan, Jul. 2018, pp. 4922–4927, doi: 10.23919/ChiCC.2018.8483841. [10] H. Myint, ‘Development of Robot Navigation System with Collision Free Path Planning Algorithm’,
Mach. Learn. Res. , vol. 3, no. 3, p. 60, 2018, doi: 10.11648/j.mlr.20180303.12. [11] H. Miao, ‘Master by Research Thesis’, p. 118. 49 [12] M. Pala, N. Eraghi, F. López-Colino, A. Sanchez, A. de Castro, and J. Garrido, ‘HCTNav: A Path Planning Algorithm for Low-Cost Autonomous Robot Navigation in Indoor Environments’,
ISPRS Int. J. Geo-Inf. , vol. 2, no. 3, pp. 729–748, Aug. 2013, doi: 10.3390/ijgi2030729. [13] A. V. Gulalkari, P. S. Pratama, G. Hoang, D. H. Kim, B. H. Jun, and S. B. Kim, ‘Object tracking and following six-legged robot system using Kinect camera based on Kalman filter and backstepping controller’,
J. Mech. Sci. Technol. , vol. 29, no. 12, pp. 5425–5436, Dec. 2015, doi: 10.1007/s12206-015-1144-4. [14] A. Z. Shukor, N. A. Natrah, A. I. Tarmizi, A. A. Afiq, and M. Herman, ‘Object Tracking and Following Robot Using Color- Based Vision Recognition for Library Environment’, vol. 10, no. 2, p. 5. [15] B. X. Chen, R. Sahdev, and J. K. Tsotsos, ‘Integrating Stereo Vision with a CNN Tracker for a Person-Following Robot’, in
Computer Vision Systems , vol. 10528, M. Liu, H. Chen, and M. Vincze, Eds. Cham: Springer International Publishing, 2017, pp. 300–313. [16] S. Sun, N. An, X. Zhao, and M. Tan, ‘Human recognition for following robots with a Kinect sensor’, in , Qingdao, China, Dec. 2016, pp. 1331–1336, doi: 10.1109/ROBIO.2016.7866511. [17] M. S. Sumbal, ‘Environment Detection and Path Planning Using the E-puck Robot’,
Int. Res. J. Eng. Technol. , vol. 03, no. 01, p. 9, Jan. 2016. [18] M. Dirik, ‘Collision-Free Mobile Robot navigation using Fuzzy Logic Approach’,
Int. J. Comput. Appl. , vol. 179, no. 9, pp. 33–39, Jan. 2018, doi: 10.5120/ijca2018916085. [19] H. Omrane, M. S. Masmoudi, and M. Masmoudi, ‘Fuzzy Logic Based Control for Autonomous Mobile Robot Navigation’,
Comput. Intell. Neurosci. , vol. 2016, pp. 1–10, 2016, doi: 10.1155/2016/9548482. [20] M. Almasri, K. Elleithy, and A. Alajlan, ‘Sensor Fusion Based Model for Collision Free Mobile Robot Navigation’,
Sensors , vol. 16, no. 1, p. 24, Dec. 2015, doi: 10.3390/s16010024. [21] M. Abaee Shoushtary, H. Hoseini Nasab, and M. B. Fakhrzad, ‘Team Robot Motion Planning in Dynamics Environments Using a New Hybrid Algorithm (Honey Bee Mating Optimization-Tabu List)’,
Chin. J. Eng. , vol. 2014, pp. 1–8, 2014, doi: 10.1155/2014/901986. [22] A. S. Khusheef, ‘Investigation on the mobile robot navigation in an unknown environment’, pp. 1–152. [23] A. Marzoughi, ‘A decentralized position estimation switching algorithm to avoid a convex obstacle’, in , Dalian, China, Jul. 2017, pp. 6031–6036, doi: 10.23919/ChiCC.2017.8028316. 50 [24] A. Marzoughi, ‘Switching Navigation for a Fleet of Mobile Robots in Multi-Obstacle Regions’,
Int. J. Mech. Eng. Robot. Res. , pp. 1–5, 2018, doi: 10.18178/ijmerr.8.1.1-5. [25] A. V. Savkin and A. Marzoughi, ‘Distributed control of a robotic network for protection of a region from intruders’, in , Macau, Dec. 2017, pp. 804–808, doi: 10.1109/ROBIO.2017.8324516. [26] A. Marzoughi, ‘Maximizing the probability of intrusion detection by a fleet of mobile robots using an intelligent game theoretic approach’, in , Dalian, China, Jul. 2017, pp. 8342–8348, doi: 10.23919/ChiCC.2017.8028678. [27] A. Marzoughi, ‘Navigating a mobile robot to avoid moving obstacles using virtual source/sink force field’, in , Macau, Dec. 2017, pp. 344–349, doi: 10.1109/ROBIO.2017.8324441. [28] D. H. Won, Y. J. Lee, S. Sung, and T. Kang, ‘Integration of Vision based SLAM and Nonlinear Filter for Simple Mobile Robot Navigation’, in , Dayton, OH, USA, Jul. 2008, pp. 373–378, doi: 10.1109/NAECON.2008.4806575. [29] J.-J. Hernández-López, A.-L. Quintanilla-Olvera, J.-L. López-Ramírez, F.-J. Rangel-Butanda, M.-A. Ibarra-Manzano, and D.-L. Almanza-Ojeda, ‘Detecting objects using color and depth segmentation with Kinect sensor’,
Procedia Technol. , vol. 3, pp. 196–204, 2012, doi: 10.1016/j.protcy.2012.03.021. [30] ‘Depth based Tracking of Moving Objects using Kinect’,
Int. J. Eng. Res. , vol. 6, no. 13, p. 7, 2018. [31] P. Benavidez and M. Jamshidi, ‘Mobile robot navigation and target tracking system’, in , Albuquerque, NM, USA, Jun. 2011, pp. 299–304, doi: 10.1109/SYSOSE.2011.5966614. [32] P. Kaur, K. J. Dana, F. A. Romero, and N. Gucunski, ‘Automated GPR Rebar Analysis for Robotic Bridge Deck Evaluation’,
IEEE Trans. Cybern. , vol. 46, no. 10, pp. 2265–2276, Oct. 2016, doi: 10.1109/TCYB.2015.2474747. [33] J. Cruz-Mota, I. Bogdanova, B. Paquier, M. Bierlaire, and J.-P. Thiran, ‘Scale Invariant Feature Transform on the Sphere: Theory and Applications’,
Int. J. Comput. Vis. , vol. 98, no. 2, pp. 217–241, Jun. 2012, doi: 10.1007/s11263-011-0505-4. [34] Yin-Tien Wang, Chin-An Shen, and Jr-Syu Yang, ‘Calibrated kinect sensors for robot simultaneous localization and mapping’, in2014 19th International Conference on Methods and Models in Automation and Robotics (MMAR)