Decentralized Ability-Aware Adaptive Control for Multi-robot Collaborative Manipulation
IIEEE ROBOTICS AND AUTOMATION LETTERS. PREPRINT VERSION. ACCEPTED JANUARY, 2021 1
Decentralized Ability-Aware Adaptive Control forMulti-robot Collaborative Manipulation
Lei Yan, Theodoros Stouraitis, and Sethu Vijayakumar
Abstract —Multi-robot teams can achieve more dexterous, com-plex and heavier payload tasks than a single robot, yet effectivecollaboration is required. Multi-robot collaboration is extremelychallenging due to the different kinematic and dynamics ca-pabilities of the robots, the limited communication betweenthem, and the uncertainty of the system parameters. In thispaper, a Decentralized Ability-Aware Adaptive Control ( DA C ) isproposed to address these challenges based on two key features.Firstly, the common manipulation task is represented by theproposed nominal task ellipsoid, which is used to maximize eachrobot’s force capability online via optimizing its configuration.Secondly, a decentralized adaptive controller is designed to beLyapunov stable in spite of heterogeneous actuation constraintsof the robots and uncertain physical parameters of the objectand environment. In the proposed framework, decentralizedcoordination and load distribution between the robots is achievedwithout communication, while only the control deficiency isbroadcast if any of the robots reaches its force limits. In this case,the object’s reference trajectory is modified in a decentralizedmanner to guarantee stable interaction. Finally, we performseveral numerical and physical simulations to analyse and verifythe proposed method with heterogeneous multi-robot teams incollaborative manipulation tasks. Index Terms —Distributed robot systems, redundant robots,robust/adaptive control, manipulation planning, mobile manip-ulation.
I. I
NTRODUCTION
Collaboration with other agents can often be beneficial. Forexample, a multi-robot team like the one shown in Fig. 1 ismore dexterous and robust in heavy and large object manip-ulation tasks [1] than a single robot. Also, in human-robotcollaboration scenarios [2], the human’s input can improve theintelligence and adaptability of the team. Yet, collaboration isnot trivial, due to the effects of one agent’s actions on theplanning, control and decision of others.Here, we investigate multi-robot collaborative manipulationtasks, where a decentralized robot team needs to achievea common objective, while each robot has different motionand force capabilities. To perform such collaborative tasks,
Manuscript received: October, 15, 2020; Revised December, 31, 2020;Accepted January, 25, 2021. This letter was recommended for publication byAssociate Editor M. Ani Hsieh and Editor A. M. Okamura upon evaluation ofthe reviewers’ comments. This work is supported by EPSRC UK RAI Hub inFuture AI and Robotics for Space (FAIR-SPACE: EP/R026092/1), ShenzhenInstitute of Artificial Intelligence and Robotics for Society and the HondaResearch Institute Europe. (Corresponding author: Sethu Vijayakumar.)
Lei Yan, Theodoros Stouraitis and Sethu Vijayakumar are with the In-stitute of Perception, Action and Behaviour, School of Informatics, Univer-sity of Edinburgh, Edinburgh, EH8 9AB, U.K. (e-mail: [email protected],[email protected], [email protected]).Sethu Vijayakumar is a visiting researcher at the Shenzhen Institute forArtificial Intelligence and Robotics for Society (AIRS).Digital Object Identifier: see top of this page. ForcepolytopeRobot A Robot BRobot CFig. 1: Pictorial description of the multi-robot collaborative manipulationsetup, where the ability of each robot is illustrated as a force polytope. each robot within the team should maximize its contributionto the task, appropriately distribute the load among otherrobots, and adapt its behaviour according to the capabilitiesof the other robots in the team. Traditional centralized controlmethods have been used for multi-robot collaboration, butassume access to an accurate model of the robot team and fullobservability of the state of the other robots and the object.However, when considering the characteristics of real-worldmulti-robot teams [3], the most pressing problems are: (i)heterogeneity of the robots’ capabilities, (ii) uncertainty of thesystem’s physical parameters and (iii) lack of high bandwidthcommunication between the robots.Therefore, we propose a method to maximize the forcecapability of each robot while designing a decentralizedadaptive controller. Using this framework, we achieve theshared manipulation task under modelling uncertainties, inputconstraints and band-limited communication.
Motion and force capabilities:
A manipulability metric [4]was first proposed as a measure of the capability of roboticmechanisms, and has been broadly used for redundancy con-trol [5]. Utilizing the task-oriented manipulability measure,the optimal joint configuration of a redundant manipulatorcan be determined [6]. An efficient closed-form calculationof the task space manipulability was presented for a 7-DOFmanipulator [7]. For a multiple-arm system, the task-spaceforce and velocity manipulability ellipsoids were given in [8].Based on a study of the dynamic manipulability of robots,two physically meaningful choices for weighting matrix wereprovided [9]. To simplify the calculation of the dynamic ma-nipulability, the weighted manipulability ellipsoid can be usedto approximate the manipulability polytope [10]. In this paper,the weighted force manipulability ellipsoid (WFME) will beadopted to optimize the force polytope of the manipulator. a r X i v : . [ c s . R O ] F e b IEEE ROBOTICS AND AUTOMATION LETTERS. PREPRINT VERSION. ACCEPTED JANUARY, 2021 . . . . . .. . .Robot controller EmergencybroadcastCommon task Object’s trajectoryInitialpose FinalposeNominal task ellipsoidManipulabilityoptimizationAbility-awareadaptive controlJointmotions ForcelimitsDesiredforces Robot KRobot 1Joint torques Joint positionsRobot iObjectmotionNominalellipsoid Common taskManipulabilityoptimizationAbility-awareadaptive controlRobot controllerJointmotions ForcelimitsDesiredforcesObjectmotionNominalellipsoid
Fig. 2: Flowchart of decentralized ability-aware adaptive control.
Redundancy exploitation:
For dexterous manipulation [11],[12], redundant manipulators were adopted to enlargeworkspace, avoid singularities and collisions with the environ-ment. Further, manipulator’s redundancy can also be exploredto maximize manipulability [13]. The stiffness feasibility re-gions of redundant manipulators and a global task-orientedstiffness optimization were used to select robot poses [14] andconfigurations [15] for force and stiffness control, respectively.This highlights the important role of the robot’s configurationin interaction tasks. The configuration of a redundant robotwas also optimized while manipulating an object in space, tominimize disturbances on its base [16]. Geometry-aware meth-ods were used to provide a manipulability-based redundancyresolution [17], [18], which enabled tracking of manipulabilityellipsoids. Yet, the desired ellipsoid is either pre-defined by ahuman expert or pre-recorded from demonstrations. Here, wedefine a nominal task ellipsoid which is generated from themanipulation task automatically and present a task-orientedmanipulability optimization to match the WFME of the robotwith the nominal task ellipsoid.
Decentralized adaptive control:
A decentralized model ref-erence adaptive controller was proposed to deal with un-certain physical parameters of the collaborative multi-robotsystem [19]. For manipulation tasks with inaccurate kinematicmodel, the adaptive controller was used to handle the closedkinematic chain constraint and achieve accurate motion track-ing with minimum-norm actuation force [20]. Recently, DeepNeural Networks were adopted to model system uncertain-ties in model reference adaptive control [21]. In studies ofmultiple mobile manipulators, distributed coordination controland synchronous cooperation control were presented to dealwith time delays and switching topologies [22]. A distributedcooperation scheme was adopted for networked mobile ma-nipulators, which exploits the formation-based task allocation and task-oriented strategy [23]. A distributed impedance con-troller has also been used for collaborative manipulation withevent-triggered communication [24]. Recently, a decentralizedadaptive controller [1] for multiple collaborative mobile robotswas introduced. This controller can track the reference velocitytrajectory without a priori knowledge of the agent’s positionand payload properties. Yet, all adaptive controllers describedabove did not consider force and torque constraints (manipu-lability) of the robots.Considering a multi-robot collaborative manipulation task,a decentralized ability-aware adaptive control ( DA C ) frame-work (shown in Fig. 2) is proposed. Our method can handleboth uncertain system parameters and input constraints withoutfull communication between the robots. In the investigatedmulti-robot collaborative manipulation setup each robot hasaccess to the desired manipulation task, which is described asa nominal task ellipsoid. Each robot tracks the nominal taskellipsoid using the task-oriented manipulability optimizationmethod, while the DA C enables multi-robot coordinationwith respect to the common manipulation task. The maincontributions of this paper are summarised as follows:1) A nominal task ellipsoid is defined based on the commonmanipulation task, and it is used to optimize the forcecapability of each manipulator.2) A decentralized adaptive controller under input con-straints is designed and proven to be Lyapunov stable.3) Different heterogeneous multi-robot systems with in-put and communication constraints realize collaborativemanipulation tasks using the proposed decentralizedability-aware adaptive control that guarantees stabilityand convergence.The remainder of this paper is organized as follows. Thepreliminary work is presented in Section II. In Section III, wedefine the nominal task ellipsoid and present the task-orientedmanipulability optimization. The decentralized ability-awareadaptive control is described in Section IV. In depth analysisof the proposed method is carried out in Section V, whereseveral numerical and physical simulations of collaborativemanipulation tasks are performed. The conclusion and futurework are discussed in Section VI.II. P RELIMINARIES
A. Manipulability Ellipsoid and Force Polytope
Generally speaking, the force manipulability ellipsoid canbe used to approximately describe the force capability of themanipulator. In the simplest case, if we consider an arbitrary n -DOF manipulator robot k with the same torque limit across allits joints, we can obtain the force manipulability ellipsoid [4]using joint torque τ k such that (cid:107) τ k (cid:107) (cid:54) . This ellipsoid is asubset of all realizable forces and is defined as F Tk (cid:0) J k J Tk (cid:1) F k (cid:54) , (1)where J k is the Jacobian matrix of robot k and F k is the force(and torque) at the end-effector of robot k .However, typically manipulators have different torque limitsfor each joint, expressed as | τ ik | (cid:54) τ ik,max for i = 1 , ..., n ,where τ ik,max is the maximum torque of i -th joint of robot k . AN et al. : DECENTRALIZED ABILITY-AWARE ADAPTIVE CONTROL 3 Thus, the force polytope [10] of the manipulator is describedby 2 n bounding inequalities as − τ k,max (cid:54) J Tk F k (cid:54) τ k,max . (2)As shown on the left side of Fig. 3, the force polytopecorresponding to joint torque limits in (2) can be approximatedby the weighted force manipulability ellipsoid (WFME) as F Tk J k W Tk W k J Tk F k (cid:54) , (3)where W k = diag (cid:16) τ k,max , · · · , τ nk,max (cid:17) is a weighting matrixused to formulate the WFME. B. Object’s Dynamics
For multi-robot collaborative manipulation scenarios, theequation of motion of the object can be written as (cid:20) m o ˙ v o I o ˙ ω o + ω o × ( I o ω o ) (cid:21) = F t + F f + G o , (4)where m o and I o are the mass and inertia of the object, v o and ω o are respectively the linear velocity and angular velocityof the object, G o = [ m o g T , ] T is the gravity, F f is thelinear and rotational friction force which is modelled as F f =[ f Tl , f Tr ] T = [ − µ l v Tt , − µ r ω Tt ] T , µ l and µ r are the linear androtational friction coefficients, respectively. F t is the externalforce exerted on the object (in this paper it is task-related). Allthe variables are expressed in the world coordinate system.III. T ASK - ORIENTED N ULL - SPACE M ANIPULABILITY O PTIMIZATION
To utilize each robot’s maximum capability, its force poly-tope needs to be optimized with respect to the manipulationtask. As WFME is a conservative approximation of the forcepolytope as shown in Fig. 3, we optimize WFME instead tomaximize each robot’s force capability. To do this, we firstdefine the nominal task ellipsoid to encode the task’s forcecharacteristics. Second, we optimize the robot’s null-spacemotion to match the WFME with the nominal task ellipsoid.
A. Nominal Task Ellipsoid
The nominal task ellipsoid is defined as an ellipsoid ofrevolution, also called a prolate spheroid, whose two principleaxes have the same length, while the third principle axis is thelongest. The nominal task ellipsoid can be generated by thetransformation of unit sphere as { c e = Cc s | (cid:107) c s (cid:107) (cid:54) } , (5)where c e is the Cartesian coordinates of the nominal taskellipsoid, c s is the Cartesian coordinates of the unit sphere. C is the transformation matrix and can be calculated as C = R r R s with R s being the scaling matrix and R r beingthe rotation matrix. The scaling matrix R s is defined as R s = diag ( l x , l y , l z ) , where l x , l y , l z , are the lengths of theprinciple axes. In this paper, l x = 1 corresponds to the longestaxis that is aligned with the desired force. l y = l z = c f l x correspond to the other two axes and c f ∈ (0 , can be set Isotropic ellipsoid c f =1 F t =[3 , , c f =0 . F t =[3 , , F t =[ − , , WFMEForcepolytope c f =0 . Fig. 3: Force polytope and WFME (left). Nominal task ellipsoids correspond-ing to different manipulation tasks where F t represents the desired force alongdifferent directions (right). according to the task requirements, e.g. c f = 1 (isotropicellipsoid in Fig. 3) for manipulation tasks that require force tobe equally distributed along different directions, while c f < (other cases in Fig. 3) for manipulation tasks that require forcealong a specific direction.Given the desired force F t on the object and the unit vector a x of the longest axis, the common perpendicular vector u and the included angle φ between them, can be obtained by u = F t × a x (cid:107) F t × a x (cid:107) and φ = acos (cid:18) F t · a x (cid:107) F t (cid:107)(cid:107) a x (cid:107) (cid:19) , (6)respectively. Therefore, the rotation matrix R r —which is usedto align the longest axis with the desired force—can beobtained according to the angle-axis representation as R r = E c φ + (1 − c φ ) uu T + u × s φ , (7)where s φ = sin( φ ) , c φ = cos( φ ) , E is the × identitymatrix and u × is the skew symmetric matrix of u .From (5), we can obtain the following equation c Te M − t c e (cid:54) , with M t = CC T , (8)where the symmetric positive definite matrix M t representsthe nominal task ellipsoid. A few nominal task ellipsoids fordifferent manipulation tasks are visualized in Fig. 3. The shapeof a nominal task ellipsoid is determined by the coefficient c f , while its orientation is determined by the direction of thedesired force F t . B. Null-Space Manipulability Optimization
As multiple robots manipulate the object jointly, the end-effector of each robot can be assumed to be fixed on the objectvia the corresponding grasping point. Thus, the velocity of theend-effector of each robot ˙ x k can be derived from the velocityof the object ˙ x o as ˙ x k = (cid:20) E − r × k E (cid:21) ˙ x o = G Tpk ˙ x o , (9)where G pk is the grasp matrix of robot k , r k is the positionvector from the object’s center of mass to the grasping pointof robot k , and ˙ x o = (cid:2) v To , ω To (cid:3) T . By differentiating (9) theacceleration constraint can be obtained as ¨ x k = G Tpk ¨ x o + ˙ G Tpk ˙ x o . (10) IEEE ROBOTICS AND AUTOMATION LETTERS. PREPRINT VERSION. ACCEPTED JANUARY, 2021
At the same time, the pseudo-inverse solutions for jointvelocity ˙ Θ k and acceleration ¨ Θ k of each robot are ˙ Θ k = J † k ˙ x k and ¨ Θ k = J † k (cid:16) ¨ x k − ˙ J k ˙ Θ k (cid:17) , (11)respectively; where J † k is the pseudo-inverse of J k .In this paper, the null-space motion of the redundant ma-nipulators is used to optimize the force manipulability giventhe current WFME of robot k and a desired nominal taskellipsoid (Section III-A). By using the tensor representationand exploiting the fact that ellipsoids lie on the Riemannianmanifold of symmetric positive definite (SPD) matrices [18],the velocity-level inverse kinematics with task-oriented manip-ulability optimization can be derived as ˙ Θ k = J † k ˙ x k + (cid:16) I − J † k J k (cid:17) (cid:16) J f † M k (cid:17) T K M vec (cid:16) ˙ M fk (cid:17) . (12)The first term of the right hand side is same as in (11), whilethe second term projects the difference between the nominaltask ellipsoid and WFME to the null space motion of themanipulator. K M is the scaling matrix, vec () denotes thevectorization of symmetric matrices with Mandel notation , ˙ M fk = Log M fk M t , the Log operator is a logarithm map [25]which can find the tangent vector between two points in theSPD manifold. M fk represents the WFME of robot k and isdefined as M fk = (cid:0) J k W Tk W k J Tk (cid:1) − . (13)The force manipulability Jacobian J fM k projects the scaledrate of change of M fk to the joint velocity ˙ Θ k and it isobtained as J fM k = − ∂ M f − k ∂ Θ k × M fk × M fk ,∂ M f − k ∂ Θ k = ∂ J k ∂ Θ k × J W + ∂ J Tk ∂ Θ k × J W , (14)where J W = J k W Tk W k , × n is the n -mode tensor prod-uct [26]. The force capability optimization is achieved in thenull space of manipulation task by using the proposed task-oriented manipulability optimization (12).IV. D ECENTRALIZED A BILITY -A WARE A DAPTIVE C ONTROL
According to the force polytope shown in Fig. 3, themaximum operational force along any specific direction canbe calculated. Subsequently, the decentralized ability-awareadaptive controller ( DA C ) computes the control inputs inaccordance with the force capability of each robot. A. Force Capability
The maximum operational force of each manipulator alongthe specific task direction F t is calculated by max F k (cid:107) F k (cid:107) (15a)s.t. | H k ¨ Θ k + C k + G k + J kT F k | ≤ τ k,max (15b) F × t F k = , (15c) The Mandel representation [ ε ] (as a column-vector) of any second rank,symmetric tensor ε is defined as follows: [ ε ] = [ ε , ε , √ ε ] T . -mode and -mode: A × U T = U T A , A × V T = AV where H k is the inertia matrix of robot k , C k is the Coriolisand centrifugal force of robot k , G k is the gravity of robot k .The force polytope is defined by (15b) and can be obtainedfrom the dynamic equation of the manipulator, while (15c) isused to define the specific task direction. B. Ability-Aware Adaptive Controller
In order to track the desired trajectory of the object duringdecentralized multi-robot collaborative manipulation, we pro-pose an ability-aware adaptive controller in which the forcecapability of each robot is considered.According to (4), the ideal reference dynamics model of theobject can be written as ¨ x ∗ o = A ∗ ˙ x ∗ o + B ∗ ( F ∗ t − N cg ) , (16)where A ∗ is a Hurwitz stable matrix written as A ∗ = (cid:20) − µ l m ∗ o E − µ r I ∗− o (cid:21) , B ∗ = (cid:20) m ∗ o E I ∗− o (cid:21) , and N cg = (cid:20) − m ∗ o gω o × ( I ∗ o ω o ) (cid:21) is a stacked vector of gravity termand nonlinear term in (4), and m ∗ o and I ∗ o are the nominal massand inertia matrix of the object. Given the bounded reference(desired) trajectory ˙ x ∗ o and ¨ x ∗ o of the object, the boundedreference control input F ∗ t , which represents the external forceexerted on the object, can be computed from (16).To design the adaptive controller, the actual object’s dynam-ics model can be rewritten in the following linear form withrespect to the system state ˙ x o as ¨ x o = A ˙ x o + K (cid:88) k =1 B k ( F k − U k ) , (17)where A and B k are unknown constant matrices, A = − (cid:20) µ l m o E µ r I − o (cid:21) , B k = (cid:20) m o E I − o r × k I − o (cid:21) , F k is theinput of robot k , K is the total number of robots, U k = W ∗ Tφk Φ k is an unknown nonlinear term caused by modellinguncertainties which can be approximated by a Radial BasisFunctions (RBFs) neural network, W ∗ φk is the weight matrixfor the RBFs, Φ k is the vector of RBFs and output bias.For each robot, the adaptive control input is designed as F k = K Txk ˙ x o + K Trk F ∗ t + K Tnk N cg + W Tφk Φ k , (18)where K xk , K rk , K nk and W φk are control gain matrices.Considering the force capability of each robot, the controlinput constraints of DA C are guaranteed by positive µ -modification [27]. Therefore, the modified ideal referencedynamics model (see (16)) and the actual system dynamics(see (17)) with input constraints are rewritten as ¨ x ∗ o = A ∗ ˙ x ∗ o + B ∗ (cid:32) F ∗ t + K (cid:88) k =1 K Tfk ∆ F k − N cg (cid:33) , (19) ¨ x o = A ˙ x o + K (cid:88) k =1 B k ( F k + ∆ F k − U k ) , (20)where K fk is the control gain matrix, ∆ F k is the controldeficiency, which is described in Appendix A, and is used togenerate the adaptive augmentation for the reference model. AN et al. : DECENTRALIZED ABILITY-AWARE ADAPTIVE CONTROL 5 (a) Without optimization (b) With optimization Fig. 4: Simulation results with and without manipulability optimization.
Furthermore, by defining the tracking error e = ˙ x o − ˙ x ∗ o ,the tracking error dynamics can be obtained as ˙ e = ¨ x o − ¨ x ∗ o = A ∗ e − K (cid:88) k =1 B ∗ ˜ K Tfk ∆ F k + K (cid:88) k =1 B k (cid:16) ˜ K Txk ˙ x o + ˜ K Trk F ∗ t + ˜ K Tnk N cg + ˜ W Tφk Φ k (cid:17) , (21)where ˜ K = K − K ∗ is the error matrix of control gain and K ∗ is ideal control gain (see Appendix B).Last, the following adaptive laws are adopted in order toguarantee asymptotic tracking of the reference trajectory. ˙˜ K xk = − Γ xk ˙ x o e T P B k , ˙˜ K rk = − Γ rk F ∗ t e T P B k , ˙˜ K nk = − Γ nk N cg e T P B k , ˙˜ W φk = − Γ φk Φe T P B k , ˙˜ K fk = Γ fk ∆ F k e T P B ∗ , (22)where P is unique SPD solution of the Lyapunov equation, P A ∗ + A ∗ T P = − Q < , where Q is any SPD matrix; Γ xk , Γ rk , Γ nk , Γ φk , and Γ fk are SPD gain matrices. The proof forasymptotic stability and asymptotic tracking using the aboveadaptive laws can be found in Appendix B. C. Computed-Torque Control for Each Robot
According to the constrained operational force and theadaptive reference trajectory of the object, the controller ofeach robot can be designed as the following well knowncomputed-torque feedforward control, τ k = H k (cid:16) ¨ Θ k + K p ˜ Θ k + K v ˙˜ Θ k (cid:17) + C k + G k + J kT ( F k + ∆ F k ) , (23) where ˜ Θ k , ˙˜ Θ k are the error vectors of joint angle and jointangular velocity, and K p , K v are the corresponding gains. D. Level of Communication
In the proposed ability-aware adaptive controller, the levelsof communication between the robots in the team may varydepending on the different states of the multi-robot system. Aslong as the desired operational force of each robot is withinits force polytope, each robot can be controlled in a fullydecentralized manner. Each robot should know the object’svelocity and the grasping location on the object. However,in cases where a robot would need to exceed its capabilitiesto track the reference input—desired operational force liesoutside of the force polytope—then, the control deficiency ∆ F k of each robot should be broadcast to all the robots ofthe team. The control deficiency of all the robots will be usedin (19) to modify the unfeasible reference control input foreach robot, which results in a coordinated adaptation of therobot team. V. R ESULTS
First, we perform three ablation studies to demonstrate thebenefits of each one of the proposed components and theircombination. Second, we demonstrate the adaptation capabil-ities of the DA C framework in a decentralized manipulationsetup. Third, we validate DA C on a physical simulation,where three heterogeneous robots manipulate an object. Thecontrol frequency of all the simulations is set to 500 Hz. A. Ablation Studies1) Force Capabilities with and without Manipulability Op-timization:
In this ablation study, three 4-DOF planar robotsmanipulate an object from initial position [0, 0] m to finalposition [-0.1, -0.2] m . The maximum joint torques of robotA, B and C are set to 0.8, 0.6 and 0.6 N m , respectively. Thesimulation results without and with task-oriented manipula-bility optimization are shown in Fig. 4. Using the proposedmethod the WFME of each robot is optimized through thenull-space motion to track the nominal task ellipsoid (yellowellipsoid), and consequently the force capability (red arrow)along the specific task direction is increased.
2) Ability-Aware vs Ability-Agnostic Adaptive Controller:
The second ablation study compares the tracking performanceof the proposed ability-aware adaptive controller ( DA C ) withan ability-agnostic adaptive controller [1]. The task considersthree robots manipulating an object, with
20 kg mass and
20 kg m inertia (along z-axis), on a plane with a slidingfriction coefficient of 0.2. The adaptive controllers are initiatedwith 80 % of these values. The reference control input isset to F ∗ t = 4 ∗ [sin(0 . t ) , sin(0 . t )] T N . The maximumforce F max of each robot is set to [1.0, 1.0] N , and theconstant vector δ (see Appendix A) is set to 10 % of F max .The object’s trajectories with ability-agnostic control (greenline) and ability-aware control (blue line) are shown in Fig. 5.For the ability-agnostic adaptive controller, the control input(see Fig. 6) exceed the force constraints, hence, we limit IEEE ROBOTICS AND AUTOMATION LETTERS. PREPRINT VERSION. ACCEPTED JANUARY, 2021
Fig. 5: State of the object with ability-agnostic andability-aware controller. Fig. 6: Control input of each robot with ability-agnostic controller. Fig. 7: Control input of each robot with ability-aware controller.Fig. 8: Tracking errors of the object (1st column) and control inputs of robot A (2nd column) with and without manipulability optimization. The cyan regionsrepresent the force capability envelope of the robot without optimization while the orange regions represents the capability envelope with optimization.Fig. 9: State of the object (1st column) and control inputs of robot A (2nd column) and robot B (3rd column) and robot C (4th column). In region (cid:13) , themass of the object is increased from 20 kg to 30 kg. In region (cid:13) , the friction coefficient is decreased from 0.2 to 0.1, while in region (cid:13) , the force capabilityof robot B is reduced to 1/3 while robot C is shut off. the control input to the maximum force. This results in asignificant deviation of the object’s trajectory from the desiredone (see Fig. 5). On the other hand, given the limited capabilityof each robot in the ability-aware controller (see Fig. 7)the new reference trajectory (red dashed line) deviates fromthe original one (blue dashed line) according to the controldeficiency of all the robots as shown in Fig. 5. In this way, themulti-robot system can track the modified trajectory accuratelywith average tracking error 0.009 m / s and 0.013 m / s alongx-axis and y-axis, respectively.
3) DA C with and without Manipulability Optimization:
Here, we compare the proposed ability-aware adaptive con-trol with and without manipulability optimization. The threemanipulators of Section V-A1) are used to manipulate thesame object as in Section V-A2). The reference trajectory iscircular and is described with v x = − . ∗ π ∗ sin( π t ) m / s , v y = 0 . ∗ π ∗ cos( π t ) m / s and w z = 0 . The nominal taskellipsoid is a sphere to equally allot the force capabilities alongdifferent directions. The velocity tracking errors of the objectare shown in Fig. 8 (left). The average tracking errors alongtwo axes with manipulability optimization are (0.0138, 0.0143) AN et al. : DECENTRALIZED ABILITY-AWARE ADAPTIVE CONTROL 7 Fig. 10: A heterogeneous multi-robot team manipulates an object. The object’s circular motion is illustrated with respect to the yellow circle, which is fixedin the world frame.Fig. 11: Desired, modified reference and actual trajectory of the object (1st column), and desired (dashed line) and actual (solid line) end-effector forcesof robot A (2nd column), robot B (3rd column) and robot C (4th column). By setting robot-specific safety zones (green region), the force of each robot isguaranteed not to violate its actual limits after a few iterations. m / s , which outperforms (0.0138, 0.0301) m / s correspondingto the one without. In the right part of Fig. 8, we can observeall the control inputs along with the force limits (regions). The DA C with manipulability optimization regulates the robot’sconfiguration to achieve similar force capabilities along bothdirections, such that tracking performance is balanced in bothdirections. B. DA C on-the-fly Adaptation
Here, we show that the proposed method is capable ofhandling changes with respect to the mass of the object,the friction coefficient and is also tolerant to faults of teammembers. We consider the same circular task, where the massof the object is increased from 20 kg (white region (cid:13) ) to30 kg (green region (cid:13) ) at 30 s , the friction coefficient isdecreased from 0.2 (green region (cid:13) ) to 0.1 (blue region (cid:13) )at 50 s , and the max force of robot B is reduced to 1/3 whilerobot C is shut off (purple region (cid:13) ) at 70 s . The results areshown in Fig. 9, the control inputs of the robots increase whenthe object’s mass is increased and decrease when the frictioncoefficient is decreased. Further, it is worth noting that in thepurple region (cid:13) , the control input of robot A is increased andthe reference trajectory is adapted, due to the loss of robot Cand the reduced capability of robot B. C. DA C for Multi-Robot Collaboration
To verify the proposed DA C framework, three heteroge-neous robots, the torque-controlled Kuka-iiwa (A), the Franka-panda (B), and one position-based admittance controlled mo-bile manipulator (C), are used to manipulate an object on asurface (see in Fig. 10), where both the object’s mass, and thefriction between the object and the surface are unknown. Thetask is a circular motion and the maximum force for all therobots is limited to 10 N , while the constant δ (see Appendix A) are set to 1, 2, and 3 N , respectively. For the torque-controlled robots, the feedback gains K p , K v in (23) are set to diag ([16, 16, 16, 12, 12, 12, 12]) and diag ([0.2, 0.2, 0.2, 0.1,0.1, 0.1, 0.1]), respectively. The interaction between the objectand the robots is modelled with a spring-damper (stiffness5000 N / m and damping 500 Ns / m ). The physical simulationand visualization are developed in MATLAB Simscape andSimulink.The object’s desired and actual trajectories, and the forceprofiles of each robot are shown in Fig. 11. The referencetrajectory is modified according to the control deficiency, dueto the limits on the robots’ force capabilities. Robot C does notalways track the desired force accurately, due to the admittancecontroller. Thus, its safety zone is set larger (green area) toalways experience forces within its limits. Also, such forcedeviations introduce further unmodelled interaction forces, dueto the formed closed-chain. Yet, each robot adapts its controlgains on-the-fly to cope with these deviations too.VI. C ONCLUSION
In this paper we propose a decentralized ability-aware adap-tive control ( DA C ) framework for multi-robot collaborativemanipulation, which can handle uncertain system parameters,input constraints and band-limited communication. The keyidea is that the force capability of each robot is maximized byexploiting its null-space motion, while the designed adaptivecontroller enables decentralized coordination according to thecapability of each robot. The proposed method achieves accu-rate trajectory tracking irrespective of the low-level controllers,and can be used for heterogeneous fixed-base and mobile-basemulti-robot systems. An open challenge is the inclusion ofjoint position limits into the ability-aware adaptive controller.In our future work, we plan to use DA C for human-robotco-manipulation experiments where the access to human’scapability is not straightforward. IEEE ROBOTICS AND AUTOMATION LETTERS. PREPRINT VERSION. ACCEPTED JANUARY, 2021 A PPENDIX AC ONTROL INPUT WITH µ - MODIFICATION
The control input with µ -modification is written as ¯ F k = 11 + µ (cid:18) F k + µ F δkmax sat (cid:18) F k F δkmax (cid:19)(cid:19) = F k , | F k | (cid:54) F δkmax µ (cid:0) F k + µ F δkmax (cid:1) , F k > F δkmax µ (cid:0) F k − µ F δkmax (cid:1) , F k < − F δkmax , (24)where µ is positive design constant, F δkmax = F kmax − δ , δ is a constant vector, < δ < F kmax , and F kmax is themaximum force of robot k which is obtained from (15). Thecorresponding control deficiency can be calculated as ∆ F k = F kmax sat (cid:18) ¯ F k F kmax (cid:19) − F k . (25)A PPENDIX BL YAPUNOV STABILITY ANALYSIS
In order to match (19) and (20), we can choose the idealgain matrices K ∗ xk , K ∗ rk , K ∗ fk , and K ∗ nk according to thefollowing forms: A + K (cid:88) k =1 B k K ∗ Txk = A ∗ , K (cid:88) k =1 B k K ∗ Trk = B ∗ , − K (cid:88) k =1 B k K ∗ Tnk = B ∗ , B ∗ K ∗ Tfk = B k . (26)According to the error dynamics in (21), we consider thefollowing Lyapunov function candidate: V (cid:16) e , ˜ K (cid:17) = e T P e + K (cid:88) k =1 tr (cid:16) ˜ K Txk Γ − xk ˜ K xk (cid:17) + K (cid:88) k =1 tr (cid:16) ˜ K Trk Γ − rk ˜ K rk (cid:17) + K (cid:88) k =1 tr (cid:16) ˜ K Tnk Γ − nk ˜ K nk (cid:17) + K (cid:88) k =1 tr (cid:16) ˜ W Tφk Γ − φk ˜ W φk (cid:17) + K (cid:88) k =1 tr (cid:16) ˜ K Tfk Γ − fk ˜ K fk (cid:17) . (27)By using the adaptive law (22), the derivative of Lyapunovfunction decreases along the tracking error dynamics as ˙ V (cid:16) e , ˜ K xk , ˜ K rk , ˜ K nk , ˜ K fk , ˜ W φk (cid:17) = − e T Qe (cid:54) . (28)Therefore, given a bounded reference input, we can concludethat the system can achieve asymptotic tracking by usingBarbalat’s Lemma. R EFERENCES[1] P. Culbertson and M. Schwager, “Decentralized adaptive control for col-laborative manipulation,” in
IEEE International Conference on Roboticsand Automation (ICRA) , 2018, pp. 278–285.[2] T. Stouraitis, I. Chatzinikolaidis, M. Gienger, and S. Vijayakumar,“Online hybrid motion planning for dyadic collaborative manipulationvia bilevel optimization,”
IEEE Transactions on Robotics , vol. 36, no. 5,pp. 1452–1471, 2020. For two real column matrices a ∈ R n and b ∈ R n : tr (cid:0) ba T (cid:1) = a T b . [3] Y. Emam, S. Mayya, G. Notomista, A. Bohannon, and M. Egerst-edt, “Adaptive task allocation for heterogeneous multi-robot teamswith evolving and unknown robot capabilities,” arXiv preprintarXiv:2003.03344 , 2020.[4] T. Yoshikawa, “Manipulability of robotic mechanisms,” The Interna-tional Journal of Robotics Research , vol. 4, no. 2, pp. 3–9, 1985.[5] ——, “Manipulability and redundancy control of robotic mechanisms,”in
IEEE International Conference on Robotics and Automation , 1985,pp. 1004–1009.[6] S. Lee, “Dual redundant arm configuration optimization with task-oriented dual arm manipulability,”
IEEE Transactions on Robotics andAutomation , vol. 5, no. 1, pp. 78–97, 1989.[7] G. Huber and D. Wollherr, “Efficient closed-form task space manipula-bility for a 7-dof serial robot,”
Robotics , vol. 8, no. 4, p. 98, 2019.[8] P. Chiacchio, S. Chiaverini, L. Sciavicco, B. Siciliano et al. , “Globaltask space manipulability ellipsoids for multiple-arm systems,”
IEEETransactions on Robotics and Automation , vol. 7, no. 5, pp. 678–685,1991.[9] M. Azad, J. Babiˇc, and M. Mistry, “Effects of the weighting matrix ondynamic manipulability of robots,”
Autonomous Robots , vol. 43, no. 7,pp. 1867–1879, 2019.[10] P. Chiacchio, Y. Bouffard-Vercelli, and F. Pierrot, “Force polytope andforce ellipsoid for redundant manipulators,”
Journal of Robotic Systems ,vol. 14, no. 8, pp. 613–620, 1997.[11] L. Yan, Z. Mu, W. Xu, and B. Yang, “Coordinated compliance control ofdual-arm robot for payload manipulation: Master-slave and shared forcecontrol,” in
IEEE/RSJ International Conference on Intelligent Robotsand Systems (IROS) , 2016, pp. 2697–2702.[12] L. Yan, Y. Yang, W. Xu, and S. Vijayakumar, “Dual-arm coordinatedmotion planning and compliance control for capturing moving objectswith large momentum,” in
IEEE/RSJ International Conference on Intel-ligent Robots and Systems (IROS) , 2018, pp. 7137–7144.[13] L. Jin, S. Li, H. M. La, and X. Luo, “Manipulability optimization ofredundant manipulators using dynamic neural networks,”
IEEE Trans-actions on Industrial Electronics , vol. 64, no. 6, pp. 4710–4720, 2017.[14] A. Ajoudani, N. G. Tsagarakis, and A. Bicchi, “Choosing poses for forceand stiffness control,”
IEEE Transactions on Robotics , vol. 33, no. 6,pp. 1483–1490, 2017.[15] D. Busson, R. Bearee, and A. Olabi, “Task-oriented rigidity optimizationfor 7 dof redundant manipulators,”
IFAC-PapersOnLine , vol. 50, no. 1,pp. 14 588–14 593, 2017.[16] L. Yan, W. Xu, Z. Hu, and B. Liang, “Multi-objective configurationoptimization for coordinated capture of dual-arm space robot,”
ActaAstronautica , vol. 167, pp. 189–200, 2020.[17] L. Rozo, N. Jaquier, S. Calinon, and D. G. Caldwell, “Learningmanipulability ellipsoids for task compatibility in robot manipulation,”in
IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS) , 2017, pp. 3183–3189.[18] N. Jaquier, L. D. Rozo, D. G. Caldwell, and S. Calinon, “Geometry-aware tracking of manipulability ellipsoids.” in
Robotics: Science andSystems , 2018, pp. 1–9.[19] Y.-H. Liu and S. Arimoto, “Decentralized adaptive and nonadaptiveposition/force controllers for redundant manipulators in cooperations,”
The International Journal of Robotics Research , vol. 17, no. 3, pp. 232–247, 1998.[20] F. Aghili, “Adaptive control of manipulators forming closed kinematicchain with inaccurate kinematic model,”
IEEE/ASME Transactions onMechatronics , vol. 18, no. 5, pp. 1544–1554, 2012.[21] G. Joshi and G. Chowdhary, “Deep model reference adaptive control,”in
IEEE Conference on Decision and Control , 2019, pp. 4601–4608.[22] G.-B. Dai and Y.-C. Liu, “Distributed coordination and cooperationcontrol for networked mobile manipulators,”
IEEE Transactions onIndustrial Electronics , vol. 64, no. 6, pp. 5065–5074, 2016.[23] Y. Ren, S. Sosnowski, and S. Hirche, “Fully distributed cooperationfor networked uncertain mobile manipulators,”
IEEE Transactions onRobotics , 2020.[24] P. B. G. Dohmann and S. Hirche, “Distributed control for cooperativemanipulation with event-triggered communication,”
IEEE Transactionson Robotics , 2020.[25] X. Pennec, P. Fillard, and N. Ayache, “A riemannian framework fortensor computing,”
International Journal of computer vision , vol. 66,no. 1, pp. 41–66, 2006.[26] T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,”
SIAM review , vol. 51, no. 3, pp. 455–500, 2009.[27] E. Lavretsky and N. Hovakimyan, “Positive mu-modification for stableadaptation in the presence of input constraints,” in