Disturbances in Influence of a Shepherding Agent is More Impactful than Sensorial Noise During Swarm Guidance
Hung The Nguyen, Matthew Garratt, Lam Thu Bui, Hussein Abbass
DDisturbances in Influence of a Shepherding Agent isMore Impactful than Sensorial Noise During SwarmGuidance
Hung The Nguyen , Matthew Garratt , Lam Thu Bui , Hussein Abbass ∗ School of Engineering and Information Technology, UNSW-Canberra, University of New South Wales
Canberra, Australia. † Department of Information Technology, Le Quy Don Technical University Hanoi, Vietnam
Abstract —The guidance of a large swarm is a challengingcontrol problem. Shepherding offers one approach to guidea large swarm using a few shepherding agents (sheepdogs).While noise is an inherent characteristic in many real-worldproblems, the impact of noise on shepherding is not a well-studied problem. We study two forms of noise. First, we evaluatenoise in the sensorial information received by the shepherd aboutthe location of sheep. Second, we evaluate noise in the abilityof the sheepdog to influence sheep due to disturbance forcesoccurring during actuation. We study both types of noise in thispaper, and investigate the performance of Str¨ombom’s approachunder these actuation and perception noises. To ensure that theparameterisation of the algorithm creates a stable performance,we need to run a large number of simulations, while increasingthe number of random episodes until stability is achieved. Wethen systematically study the impact of sensorial and actuationnoise on performance. Str¨ombom’s approach is found to bemore sensitive to actuation noise than perception noise. Thisimplies that it is more important for the shepherding agentto influence the sheep more accurately by reducing actuationnoise than attempting to reduce noise in its sensors. Moreover,different levels of noise required different parameterisation forthe shepherding agent, where the threshold needed by an agentto decide whether or not to collect astray sheep is different fordifferent noise levels.
Index Terms —Multi-Agent Systems, Shepherding, SwarmRobotics, Swarm Guidance
I. I
NTRODUCTION
The research area of bio-inspired swarm control has at-tracted diverse ideas and perspectives from a variety of re-search areas, including control theory, biology, and artificialintelligence [1]. These bio-inspired approaches provide valu-able insights into designing multi-agent systems and/or roboticswarms. In these systems, studies aim to address the questionof how to control a swarm of individual agents based onnatural interactions occurring among the agents and betweenthe agents and their operating environment [2], [3].The shepherding problem is inspired by sheep-herding inagriculture wherein a single or multiple shepherds or sheep-dogs are used to guide a large group of sheep. Shepherdingis considered as a guided-flocking behaviour when one ormultiple external agents acting as shepherds, drive a swarm of individual agents, called flocking or sheep agents towards agiven target. The herding idea has been applied to the fieldof multi-agent systems and swarm robotics [4]. There aremany applications [4], [5] of the shepherding problem withinthese fields such as herding living animals such as drivinga large group of bird or sheep in a field area, assisting incontrolling human crowd activities, cleaning environmentalhazards such as oil-spills, or guiding cells to fix tissues ininternal medicine [6].In recent years, a heuristic approach, which is proposedby Str¨ombom et al. [4], [7], has addressed the shepherdingproblem effectively. In this approach, two main behavioursare used: collecting and driving , which are used to explain theinteraction between a shepherd considered as one intelligentagent and a swarm of sheep being treated as autonomousagents. The collecting behaviour aims to maintain the entiresheep grouped within a connected network, while the drivingbehaviour enables guidance of the group/swarm of sheep to-wards a goal. The Str¨ombom approach shows good shepherd-ing performance in successfully collecting and driving a largenumber of sheep towards the given target. However, Str¨ombomet al. evaluate their approach in an ideal environment possess-ing an unrealistically low noise level for both the shepherd andthe sheep. In practice, the shepherd might face various sourcesof significant noise associated with the response of the sheep tothe influence of the sheepdog, called the actuation noise , andnoise coming from the sensing ability of the shepherd, calledthe perception noise . Under extreme weather conditions, orobstacles, the sheep or autonomous agents might move veryimprecisely in response to shepherding commands, and theshepherd can inadequately sense the position of the sheep.These errors may lead to poor performance of the shepherd.In the literature, there are insufficient attempts to understandthe impact of these errors on the overall performance of theshepherding in the successful completion of the task.In this paper, we first investigate the level of evaluationssufficient to achieve stability in performance in the Str¨ombomapproach [7]. After that, we study the performance of themodel under increasing actuation and perception noises. Fur-thermore, we identify appropriate thresholds of switching a r X i v : . [ c s . M A ] O c t etween the collecting and driving behaviours, called thecollecting frequency, in order to improve the performanceof the shepherd. To identify these thresholds, we system-atically decrease and increase the threshold value used inthe Str¨ombom method. Our experiments are conducted inthe same simulation environment as that introduced in theStr¨ombom approach. The results from the experiments showthat the performance of the shepherd is more sensitive toperception noise than actuation noise. Moreover, a guidingset of appropriate thresholds are identified which should helpimprove the shepherding efficiency by adapting the switchbetween collecting and driving behaviours to suit the amountof perception and actuation noises present.The remainder of the paper is organized as follows. InSection II, we provide a brief review of the research focusedon the shepherding problem in order to identify a gap in theevaluation of shepherding performance under noise conditions.Following this section, we formally define shepherding usingan appropriate notional system and a corresponding math-ematical objective in Section III. The proposed evaluationframework is introduced in Section IV. The framework isconducted in a simulated shepherding task in Section V.Section VI presents the results of the framework. Conclusionsare drawn in Section VII, followed by a discussion on futurework. II. R ELATED WORK
In nature, the behaviours of flocking of birds, herding ofland animals, or schooling of fish can be widely seen [8]. Inthese behaviours, a large number of individual agents will beinfluenced by one controlling agent in order to successfullyachieve different goals such as finding food or foraging.Studying the various kinds of swarm behaviours in naturecan greatly assist in the design of distributed and coordinatedcontrol methods for robotic swarms or multi-agent systems [9],[10].Early research on the shepherding problem was carried outby Schultz et al. [11]. In this research, the authors use geneticalgorithms to learn rules for a shepherd or sheep-dog agent sothat it might drive a swarm of sheep towards a desired target.Lien et al. [12] conducted experiments in order to simulate fourmain behaviours: herding, covering, patrolling, and collecting.The combination of these four behaviours for the shepherdshows effective shepherding strategies. However, both piecesof research are more suitable for driving a small number ofsheep (less than 40) [13].Towards guiding a larger number of sheep agents (morethan 40), Str¨ombom et al. [4], [7] used a heuristic approach tochoose an appropriate threshold to switch between collectingand driving behaviours in order to guide the entire sheep.The approach is promising, enabling the shepherd to guideup to 300 sheep effectively. Adopting Str¨ombom approach,some other research has attempted to use artificial intelligencemethods, such as reinforcement learning [14], apprenticeshiplearning [15], and machine education [16], [17]. However,in both the Str¨ombom method and the research adopting the approach of Str¨ombom, the shepherd works in an idealenvironment with just a small amount of noise added to theshepherd and the sheep to avoid deadlocks. In practice, theoperating environment of these agents might include varioussignificant noise sources impacting on the performance of theshepherd. One source of noise comes from responses andbehaviours of the sheep that deviate from the intent of thesheepdog due to differences between intent and actuation, oractuation and response, we group them under actuation noise.Another source of noise is the sensing ability of the shepherd,called the perception noise. To date, there has not been anypublished work on contrasting these noises on the performanceof a shepherd for swarm guidance.III. S
HEPHERDING P ROBLEM
In the shepherding problem, Str¨ombom et al. [7] introducea heuristic approach in which the movement of sheep iscomputed, and from there an effective control strategy iscreated for the shepherd. In this paper, the Str¨ombom et al. [7]approach is described by providing the notations as well as themechanism that we will use later in the experimental design.The operating environment of the shepherding problem is a2-D square paddock having length of L . In this environment,two kinds of agents, which are a set of sheep (called influencedagents) Π = { π , . . . , π i , . . . , π N } , and a set of shepherds(called influencing agents) B = { β , . . . , β j , . . . , β M } , areinitialized. There are three main behaviours for each shepherd,and four basic behaviours for each sheep at a time step t . Thesebehaviours are shown as below.1) For shepherd β j : • Driving behaviour σ : When all sheep are collectedin a cluster, i.e. all the distances from the observedsheep to the center of sheep’s mass are lower than athreshold f ( N ) calculated in Equation 1, a normal-ized force vector, F tβ j cd , is applied for the shepherdas a velocity vector in order to reach a driving point.This point is located behind the sheep’s mass on theline drawn from the center of the sheep’s mass andthe target position. The distance from the center ofthe mass to the driving point is R ππ × √ N . f ( N ) = R ππ N (1)where R ππ is the sensing range among sheep. • Collecting behaviour σ : When a sheep is deemedto have gone astray from the others i.e. the distancefrom the sheep to the center of the sheep’s massis greater than the threshold f ( N ) , a normalizedforce vector, F tβ j cd , is applied for the shepherd as avelocity vector in order to reach a collecting point.This point is positioned behind the outer or furthestsheep on the line drawn from the center of thesheep’s mass to the furthest sheep. • Jittering behaviour σ : To avoid an impasse duringmoving, a small random noise F tβ j (cid:15) with weight W eβ j , is added to the total force.he total force F tβ j of the shepherd β j (total forcebehaviour σ ) is a weighted combination of the forcesproduced by the driving/collecting behaviour and the jit-tering behaviour. This total force is shown in Equation 2 F tβ j = F tβ j cd + W eβ j F tβ j (cid:15) (2)2) For sheep π i : • Escaping behaviour σ : This behaviour happenswhen the distance between the sheep π i at position P tπ i and the shepherd β j at position P tβ j is less thanthe sensing range, R πβ , a repulsive force F tπ i β j isprovided the sheep π i . The condition to trigger thebehaviour is shown in Equation 3. (cid:107) P tπ i − P tβ j (cid:107) ≤ R πβ (3) • Collision avoidance behaviour σ : This behaviourhappens when there is a repulsion between the sheep π i and the other sheep π k (cid:54) = i . The condition ofactivating the repulsion force between the two sheepis that the distance between them is less than thesensing range among sheep, R ππ . This condition isshown in Equation 4. ∃ k, such that (cid:107) P tπ i − P tπ k (cid:107) ≤ R ππ (4)Then, we have the summed force vectors, F tπ i π − i ,from all the other sheep within the threshold range, R ππ , applied onto sheep π i . • Grouping behaviour σ : This behaviour appearswhen the sheep π i under a force F tπ i Λ tπi will beattracted to move towards the center of the mass ofits sheep neighbors, Λ tπ i . • Jittering behaviour σ : Similar to the jittering be-haviour of each shepherd, to avoid impasse, a smallrandom noise is added to the total force F tπ i (cid:15) withweight W eπ i .The total force, F tπ i , of the sheep π i is represented by aweighted sum of individual force vectors F tπ i β j , F tπ i π − i , F tπ i Λ tπi , and F tπ i (cid:15) ; that is, F tπ i = W π υ F t − π i + W π Λ F tπ i Λ tπi + W πβ F tπ i β j + W ππ F tπ i π − i + W eπ i F tπ i (cid:15) (5)The shepherds’and sheep’s positions are calculatedaccording to Equations 6, and 7. Meanwhile, the given S tβ j and S tπ i are the speed of the shepherd β j and the speed of thesheep π i at time step t . In the original Str¨ombom approach,the speeds of both the shepherds and sheep are constant. P t +1 β j = P tβ j + S tβ j F tβ j (6) P t +1 π i = P tπ i + S tπ i F tπ i (7) IV. A P ROPOSED E VALUATION F RAMEWORK .In this paper, we evaluate the performance of the shepherdproduced by the Str¨ombom approach [7] under the two noises:the actuation ( λ ) and perception ( α ). The actuation noiseappears when the sheep move randomly around the locationthey are supposed to move to. The range of the randommovement decides the degree of this noise. Meanwhile, theperception noise happens when there is deviation betweenthe sheep’s actual position and the position that the shepherdobserves. The deviation range between these two positionsdefines the degree of the perception noise.The standard normal distribution, with a mean of zero andstandard deviation of 1, is used to create the actuation andperception noises. The procedure of updating the position ofsheep π i under the actuation noise, called ActP t +1 π i , is shownin Equation 8. ActP t +1 π i = P tπ i + S tπ i × ( F tπ i + λ × StandardN ormal ()) (8)For the perception noise, the perceived position of sheep π i at timestep t + 1 is denoted P erP t +1 π i . This position is sensedby the shepherd by Equation 9. P erP t +1 π i = ActP t +1 π i + α × StandardN ormal () (9)According to the Str¨ombom approach [7], there are twomain behaviours: collecting and driving. To switch betweenthese behaviours, the shepherd needs to check whether anysheep is further from the center of mass than the threshold( f ( N ) ) as calculated in Equation 1. We aim to identify appro-priate thresholds ( f ( N ) ) of triggering between the collectingand driving behaviours, informed by the estimates of noiselevels, in order to improve the performance of the shepherd.To find these appropriate thresholds, we attempt valuesabove and below the threshold of the Str¨ombom approachshown in Equation 1 by decreasing or increasing a constant,called ∆ f . In our evaluation, we conduct three decreasinglevels of this constant ( − , − , − ) and similarly threeincreasing levels of this constant ( , , ). We refer to theconstant as a threshold due to its impact on Str¨ombom’sthresholding function to switch behaviour. These six levelswill be multiplied with the ∆ f . Thus, we have seven thresholdvalues from f ( N ) − × ∆ f to f ( N ) + 3 × ∆ f as shownin Table I. It can be understood that when the thresholdvalue increases, the collecting frequency will decrease,and the shepherd focuses on the driving behaviour. We set ∆ f = 5( meter ) in this paper.For both actuation and perception noises, we set six noiselevels increasing . from . to . and . from . to . , respectively. To add these noises to the operation, wemultiply these noise levels with a fixed change value, ∆ n ,which is set to the investigated maximum threshold value f +3 .As well as the six noise levels for the perception noise, wealso investigate the performance of the shepherd in the sameABLE I: The Threshold to Switch Between Collecting andDriving. Collection Frequency Parameter Value (meter)Extreme f − f ( N ) − × ∆ f Very High f − f ( N ) − × ∆ f High f − f ( N ) − × ∆ f Normal f f ( N ) Infrequent f +1 f ( N ) + 1 × ∆ f Very Infrequent f +2 f ( N ) + 2 × ∆ f Rare f +3 f ( N ) + 3 × ∆ f perception condition of the Str¨ombom model [7] without noise.Hence, we have seven noise levels for both as given in Table IIand III. TABLE II: Levels of Perception Noise ( α ) Level Perception Noise Value (meter)Noise Free α Very little α . × ∆ n Little α . × ∆ n Small α . × ∆ n Medium α . × ∆ n High α . × ∆ n Very High α . × ∆ n TABLE III: Levels of Actuation Noise ( λ ). Level Actuation Noise Value (meter)Noise Free λ Very little λ . × ∆ n Little λ . × ∆ n Small λ . × ∆ n Medium λ . × ∆ n High λ . × ∆ n Very High λ . × ∆ n In this work, the perception noise values are considerablyhigher than that of the actuation noise. The reason is becauseunder actuation noise, the sheep also have repulsion andattraction forces among them; thus, the movement of the sheepunder the actuation noise is more spread. Meanwhile underthe perception noise, the shepherd has a large sensing range(65 meters in the Str¨ombom approach), and then it is sillable to control the sheep acceptably without reaching thetrue collecting and driving points. Therefore, the perceptionnoise values need to be larger in order to assess its impact onperformance.The performance (
P F ) of the shepherd is validated basedon combining the three above factors: the changing radius ofthe mass ( f ), and the two noise conditions ( λ and α ). Thisrelation is illustrated in Equation 10 in which the function - g includes three variables ( f, λ, α ). P F β i = g ( f, λ, α ) (10)with β i is the shepherd i -th. In the next section, we providethe design of the experiments in this paper.V. E XPERIMENTS
In this paper, we simulate the environment given by theStr¨ombom model [7] as introduced in Section III. The same TABLE IV: Environmental parameters in the simulation.
Parameter Meaning Value L Length and Width of Environment 150 N Number of Sheep 100 M Number of Shepherds 1 R πβ Sensing range of a sheep for the shep-herd 65 R ππ Sensing range of a sheep for anothersheep 2 W ππ Sheep repulsion strength from othersheep 2 W πβ Sheep repulsion strength from the shep-herd 1 W π Λ Sheep attraction strength to sheep centreof mass 1.05 W π υ Inertial strength of sheep previous direc-tion 0.5 W eπ i Strength of sheep movement noise 0.3 W eβ j Strength of the shepherd movementnoise 0.3 | Ω π i π | Number of sheep (neighborhood) asheep can sense 25 S π Maximum speed of sheep 1 S β Maximum speed of the shepherd 2 D Minimum distance between the sheep’sglobal centre of mass and the target forsuccessful mission 5 parameters regarding the environment initialization and theinteraction between agents for the simulation are listed inTable IV.In each simulation, a given number of sheep are randomlyinitialized at the centre of the paddock with their coordinatesranging between 1/4 and 3/4 of the length/width of theenvironment. The shepherd is randomly initialised at the lowerleft corner, with their coordinates not exceeding 1/10 of thelength/width of the environment, near the target position (at (0 , ). The shepherd’s mission is to collect outer sheep intoa group and herd the entire sheep towards the target. Themission is achieved when all of the sheep have reached thetarget within a limit of 1000 steps, and the simulation ends.The illustration of the environment is shown in Figure 1.Fig. 1: The experiment environment. A. Experimental Setups
In this paper, we evaluate the performance of the shepherdof the Str¨ombom model [7] under the two noises: actuation λ and perception - α . Furthermore, we aim to identify theappropriate thresholds ( f ) of triggering between the collectingand driving behaviours that might lead to higher performancefor the shepherd under these two noises. Thus, in total weconduct × × setups in which there are 7 changinglevels of the threshold ( f ), 7 perception noise levels ( α ), and7 actuation noise levels ( λ ). B. Evaluation Metrics
In order to evaluate the performance of the shepherd underthese three factors ( f, λ, α ), we use two assessment metrics asbelow: • Number of steps (NS) : the number of time steps for thesheep to be herded to the target location. • Success rate ( % ) (SR) is the percentage of missionscompleted computed on a number of testing cases. Themission success is achieved when all the sheep arecollected and driven to the goal position. • Standard Error of Mean (SEM) of NS indicatesstability of the evaluation. This metric is calculated inEquation 11.
SEM = Std/ √ n (11)where Std is the standard deviation of the number ofsteps, and n is the number of episodes. • Standard Error Percentage (SEM-P) of Mean indicateswhich episode the evaluation should be stopped when theSEM-P is less than a small threshold (in this work, wechoose 3 % ) of the mean.VI. R ESULTS AND D ISCUSSION
In this paper, we conduct 300 random testing episodesfor each of the 343 setups. This number of testing episodesenables our evaluations to be able to reach a level of stabilityto ensure appropriateness of the analyses and precision inassessing the performance of the shepherd. The evaluationachieves our stability criterion when the SEM-P value of theperformance of the shepherd in setups is less than the thresholdof 3 % of the mean.Firstly, we investigate the effects of the actuation ( λ ) andperception ( α ) noises on the performance of the shepherd. Weshow the evidences of the stability of the evaluation whenthe values of the SEM and SEM-P of the setups decreasegradually and maintain stability by the end of the 300 episodes.According to the SEM values of f , Figure 2 shows thatwhen there is no actuation noise ( λ ), the performance of theshepherd is stably evaluated and maintained after 150 episodesat pairs of λ and the perception noise levels α lower than thehigh level. At the high ( α ) and very high ( α ) perceptionnoise levels, the stability as measured by the performance ofthe shepherd is not obtained, and then, the reliability of ourability to estimate the performance might be imprecise. Theperformance instability at the high and very high levels of theperception noise can be seen in Figure 3. In this Figure, wecan see that just only two setups of the very infrequent f +2 and the rare of collecting frequency f +3 at the high level ( α ), and the setup of the rare of collecting frequency f +3 at thevery high level ( α ) have SEM-P values below the stop pointof 3 % .Similarly, according to the SEM values of f , Figure 4 showswhen there is no perception noise, the shepherd exhibits stablebehaviour over 150 episodes at pairs of λ from the noise freelevel to the medium level and α . When high actuation noise( λ ) is applied, the behaviour of the shepherd fluctuates morein the first 150 episodes, and takes another 150 episodes toreach to the stable point wherein all the SEM-P values of thethreshold f reduces to below 3 % can be seen in Figure 5.Furthermore, with the very high actuation noise ( λ ), theshepherd is not able to achieve the mission.Similar results coming from the other pairs of λ and α show that the stability measured from performance of theshepherd is not reached at noise levels higher than medium inboth actuation and perception. Especially, the shepherding taskcollapses at a very high level of actuation noise λ . Figure 6illustrates this instability when the majority of the SEM andSEM-P values show instability by the end of our evaluationand can not reduce to a stable point wherein the SEM-P valuesare below . We note that in the unstable condition of theevaluation, the reliability to estimate the performance of theshepherd might be imprecise so in the next parts, we just focuson analysing the performance on the setups reaching stability.After validating the stability of the setups, we investigatehow the actuation and perception noises impact the perfor-mance of the shepherd. Figure 7 shows actuation noise ( λ )impacting the performance of the shepherd more dramaticallythan the perception noise ( α ). We can see that when underthe noise-free condition of perception α , the shepherdingtask collapses at λ ; meanwhile, without the actuation noise( λ ), the shepherd is still able to successfully achieve theshepherding task until the very high level- α is reached withapproximately 600 steps. It notes that the value of the actuationnoise is considerably smaller than that of the perception noise(10 times). Furthermore, it is interesting that the change of thethreshold f does not impact drastically on the performanceof the shepherd under the actuation noise; meanwhile, forthe perception noise, this change has obvious effects on theshepherd’s performance. With decreasing collecting frequency,the success rate is maintained at nearly and a smallernumber of steps are required even though the perception noiseincreases.Besides evaluating the effect of the two types of noises onthe performance of the shepherd, we conduct an additionalevaluation between the changes of the collecting frequency andthe performance. From this evaluation, a set of the appropriatethresholds f leading to the shepherd’s higher performance isprovided in this paper. We focus on those setups achieving astable performance. These setups vary in noise from very littlenoise ( λ and α ) to the medium noise ( λ and α ). Figure 8shows the evidences of obtaining these appropriate thresholds.It can be seen that for the very little noise level of the actuationas shown in Figure 8(a), the decrease in collecting frequencyleads to considerably higher performance as well as the smaller a) SEM of f at λ and α (b) SEM of f at λ and α (c) SEM of f at λ and α (d) SEM of f at λ and α (e) SEM of f at λ and α (f) SEM of f at λ and α (g) SEM of f at λ and α Fig. 2: Standard Error of Mean to Evaluate Stability of Alpha ( α ) with Different Thresholds or Collecting Frequency ( f ) atLambda ( λ ) in 300 Episodes. (a) SEM-P of Mean of f at ( λ , α ) (b) SEM-P of f at ( λ , α ) Fig. 3: Standard Error of Mean to Evaluate Stability of Alpha( α ) with Different Thresholds or Collecting Frequency ( f ) atLambda ( λ ) in 300 Episodes.number of steps when the perception noise increases. However,when the level of the actuation noise increases, under the littlenoise level of the perception as illustrated in Figure 8(b), theshepherd should prefer the extreme and very high collectingfrequency in order to have a higher success rate of nearly compared to approximately of the Str¨ombom approacheven though it takes more steps. Under the higher noise level ofperception, the decreasing collecting frequency will improveperformance in terms of both the success rate and the numberof steps.Similarly, Figure 8(c) shows the case of having the smallnoise level ( λ ) of the actuation, the trend of choosing thesmall threshold f under the very little or little perceptionnoises allows an improvement in performance when compar-ing it with the Str¨ombom approach. Furthermore, at a lowperception noise level, it seems that the Str¨ombom approach produces the best performance. Additionally, when the actu-ation noise is at a medium level, there is not considerabledifference between the performance under the increasing per-ception noise and the thresholds f . It can be understood thatthe ability of the shepherd to guide the sheep deteriorates underlarger noises, and instability is nearly reached.From this evaluation, it is interesting to demonstrate therobustness of the shepherding model against both sources ofnoise when the testing scenarios at α , α , and λ could stillhave a 20% minimum success rate. Regarding the changesof the threshold ( f ), when the noises increase, it is logicalthat this threshold should be increased to prefer the drivingbehaviour and reduce the collecting behaviour. Under largenoises, there appears more sheep going out of the mass, and theshepherd might perform the collecting behaviour continuously,and then there is no chance to drive the sheep towards thetarget. This is a possible reason to see the task collapsing atthe larger noise and the smaller threshold radius.VII. C ONCLUSION AND F UTURE W ORK
In this paper, we evaluate the performance of a shepherdguiding a swarm under actuation and perception noises. With300 random episodes for 343 setups, the obtained results showthat stability in performance is reached and maintained afterthe first 150 episodes at noise levels not exceeding the highlevel identified for both actuation and perception. When thenoises are at high levels, the stability breaks down, and thenthe reliability of our ability to estimate the performance isvery likely to become imprecise. After validating stability, avaluable point is drawn that the actuation noise has a higherimpact on the performance of the shepherd than perception a) SEM of f at λ and α (b) SEM of f at λ and α (c) SEM of f at λ and α (d) SEM of f at λ and α (e) SEM of f at λ and α (f) SEM of f at λ and α Fig. 4: Standard Error of Mean to Evaluate Stability of Lambda ( λ ) with Different Thresholds or Collecting Frequency ( f ) atAlpha ( α ) in 300 Episodes. (a) SEM-P of Mean of f at λ and α (b) SEM-P of Mean of f at λ and α Fig. 5: Standard Error of Mean to Evaluate Stability of Lambda( λ ) with Different Thresholds or Collecting Frequency ( f ) atAlpha ( α ) in 300 Episodes. (a) SEM of f at ( λ , α ) (b) SEM-P of f at ( λ , α ) Fig. 6: Standard Error of Mean to Evaluate Stability of Alpha( α ) with Different Thresholds or Collecting Frequency ( f ) atLambda ( λ ) in 300 Episodes.noise. The performance of the shepherd deteriorates earlier at (a) NS of f at α in 300 Episodes (b) NS of f at λ in 300 Episodes(c) SR of f at α in 300 Episodes (d) SR of f at λ in 300 Episodes Fig. 7: The Relationship between Lamda ( λ ), Alpha ( α ), andthe Threshold or the Collecting Frequency ( f ) in 300 Episodeswhen the Standard Error Percentage of Mean is below 3percent (%)a higher level of actuation noise, though this noise’s value isless than one tens of the perception noise causing the samelevel of degradation in performance.Additionally, when the perception noise increases and theactuation noise is low, the lower collecting frequency leads to a) SR of α at λ (b) SR of α at λ (c) SR of α at λ (d) SR of α at λ (e) NS of α at λ (f) NS of α at λ (g) NS of α at λ (h) NS of α at λ Fig. 8: The Effects of Lamda ( λ ) and Alpha ( α ) on Different Thresholds or Collecting Frequency ( f ) in 300 Episodes whenthe Standard Error Percentage of Mean is below 3 percent (%).higher success rate. In contrast, when the actuation noise ishigher and the perception noise is low, the higher collectingfrequency contributes to higher success rate. These interest-ing results show promising evidences in order to design anadaptive behaviour controller, which allows the threshold f to switch between the two collecting and driving behavioursto be adjusted, improving the performance of the shepherdunder these noises. Our future work attempts to design thiscontroller. R EFERENCES[1] S. Martinez, J. Cortes, and F. Bullo, “Motion coordination with dis-tributed information,”
IEEE Control Systems Magazine , vol. 27, no. 4,pp. 75–88, 2007.[2] R. Carelli, C. De la Cruz, and F. Roberti, “Centralized formation controlof non-holonomic mobile robots,”
Latin American applied research ,vol. 36, no. 2, pp. 63–69, 2006.[3] H. Oh, A. R. Shirazi, C. Sun, and Y. Jin, “Bio-inspired self-organisingmulti-robot pattern formation: A review,”
Robotics and AutonomousSystems , vol. 91, pp. 83–100, 2017.[4] D. Str¨ombom and A. J. King, “Robot collection and transport of objects:A biomimetic process,”
Frontiers in Robotics and AI , vol. 5, p. 48, 2018.[5] P. Nalepka, R. W. Kallen, A. Chemero, E. Saltzman, and M. J. Richard-son, “Practical applications of multiagent shepherding for human-machine interaction,” in
International Conference on Practical Applica-tions of Agents and Multi-Agent Systems . Springer, 2019, pp. 168–179.[6] D. J. Cohen, W. J. Nelson, and M. M. Maharbiz, “Galvanotactic controlof collective cell migration in epithelial monolayers,”
Nature materials ,vol. 13, no. 4, p. 409, 2014.[7] D. Str¨ombom, R. P. Mann, A. M. Wilson, S. Hailes, A. J. Morton, D. J.Sumpter, and A. J. King, “Solving the shepherding problem: heuristicsfor herding autonomous, interacting agents,”
Journal of the royal societyinterface , vol. 11, no. 100, p. 20140719, 2014.[8] C. W. Reynolds,
Flocks, herds and schools: A distributed behavioralmodel . ACM, 1987, vol. 21, no. 4.[9] J. Kennedy and R. Eberhart, “Particle swarm optimization,” in
NeuralNetworks, 1995. Proceedings., IEEE International Conference on , vol. 4.IEEE, 1995, pp. 1942–1948. [10] R. Eberhart and J. Kennedy, “A new optimizer using particle swarmtheory,” in
Micro Machine and Human Science, 1995. MHS’95., Pro-ceedings of the Sixth International Symposium on . IEEE, 1995, pp.39–43.[11] A. Schultz, J. J. Grefenstette, and W. Adams, “Roboshepherd: Learninga complex behavior,”
Robotics and Manufacturing: Recent Trends inResearch and Applications , vol. 6, pp. 763–768, 1996.[12] J.-M. Lien, O. B. Bayazit, R. T. Sowell, S. Rodriguez, and N. M. Amato,“Shepherding behaviors,” in
IEEE International Conference on Roboticsand Automation, 2004. Proceedings. ICRA’04. 2004 , vol. 4. IEEE,2004, pp. 4159–4164.[13] B. Bennett and M. Trafankowski, “A comparative investigation ofherding algorithms,” in
Proc. Symp. on Understanding and ModellingCollective Phenomena (UMoCoP) , 2012, pp. 33–38.[14] H. T. Nguyen, T. D. Nguyen, M. Garratt, K. Kasmarik, S. Anavatti,M. Barlow, and H. A. Abbass, “A deep hierarchical reinforcement learnerfor aerial shepherding of ground swarms,” in
International Conferenceon Neural Information Processing . Springer, 2019, pp. 658–669.[15] H. T. Nguyen, M. Garratt, L. T. Bui, and H. Abbass, “Apprenticeshiplearning for continuous state spaces and actions in a swarm-guidanceshepherding task,” in . IEEE, 2019, pp. 102–109.[16] A. Gee and H. Abbass, “Transparent machine education of neural net-works for swarm shepherding using curriculum design,” arXiv preprintarXiv:1903.09297 , 2019.[17] N. R. Clayton and H. Abbass, “Machine teaching in hierarchical geneticreinforcement learning: Curriculum design of reward functions forswarm shepherding,” arXiv preprint arXiv:1901.00949arXiv preprint arXiv:1901.00949