A Dual-arm Robot that Autonomously Lifts Up and Tumbles Heavy Plates Using Crane Pulley Blocks
PPREPRINTS AT ARXIV, SUBMITTED TO A JOURNAL FOR REVIEW, 2020 1
A Dual-arm Robot that Autonomously Lifts Up andTumbles Heavy Plates Using Crane Pulley Blocks
Shogo Hayakawa , Weiwei Wan ∗ , Keisuke Koyama and Kensuke Harada , Abstract —This paper develops a planner that plans the actionsequences and motion for a dual-arm robot to lift up and flipheavy plates using crane pulley blocks. The problem is motivatedby the low payload of modern collaborative robots. Insteadof directly manipulating heavy plates that collaborative robotscannot afford, the paper develops a planner for collaborativerobots to operate crane pulley blocks. The planner assumes atarget plate is pre-attached to the crane hook. It optimizes dual-arm action sequences and plans the robot’s dual-arm motionthat pulls the rope of the crane pulley blocks to lift up theplate. The crane pulley blocks reduce the payload that eachrobotic arm needs to bear. When the plate is lifted up to asatisfying pose, the planner plans a pushing motion for one ofthe robot arms to tumble over the plate while considering forceand moment constraints. The article presents the technical detailsof the planner and several experiments and analysis carried outusing a dual-arm robot made by two Universal Robots UR3 arms.The influence of various parameters and optimization goals areinvestigated and compared in depth. The results show that theproposed planner is flexible and efficient.
Index Terms —Manipulation Planning, Grasping, Grippers andOther End-Effectors
I. I
NTRODUCTION A LTHOUGH modern industrial robots can lift as muchas 1000 kg payload. They tend to be expensive, bulky,dangerous, and have to be fenced in a work cell to keephuman workers safe. They are not suitable for human-intensivemanufacturing sites. On the other hand, collaborative robotsare developed to work together with humans, they have asmaller size and are considered safe. However, they tend tohave small payloads to ensure a good response to collisions.The heavy load and collaboration form a trade-off that leads toan interesting and unsolved problem – how to use collaborativerobots to manipulate heavy objects.Previously, researchers in robot manipulation suggestedworking on heavy objects using non-prehensile manipulation,which leverages external supports to afford a part of the totalworkload [1][2][3]. Despite the cleverness, the masses of thetarget objects to be manipulated remain limited. They mustmeet the force requirements from external supports. Also,carrying out complicated manipulation tasks like turning overthe object is difficult and deserves sophisticated optimization,and planning [4][5]. Under this background, we explore newmanipulation methods to work on heavy objects using collab-orative robots. Graduate School of Engineering Science, Osaka University, Japan. National Inst. of AIST, Japan.Contact: Weiwei Wan, [email protected]
Fig. 1: Human workers in a sewage press machine factory flippress boards with the help of a gantry crane. (a) Attaching theboard to the crane hook using bearing belts. (b) Activatingthe crane to lift up the board. (c) The board is lifted up to asatisfying pose. (d) The board is flipped and is returned to theworkbench.The conception we have in mind is to take robots as humansand plan them to manipulate various objects using humans’tools. Especially for heavy objects, we propose to use cranepulley blocks as the tool. The policy is inspired by a productionprocess seen in a factory that produces sewage press machines.Sewage press machines are used to dehydrating coals. Thepressboard of sewage press machines could be as heavy as1000 kg . In a factory that produces sewage press machines,as is shown in Fig.1, human workers need to flip and cleanboth sides of the board before installing them to the cylinderaxis. The factory developed a production procedure wherehuman workers perform the flipping task using a gantry crane.They attach the board to the crane hook using bearing belts,activate the crane to lift up the board. When the board israised to a satisfying pose, as is shown in Fig.1(c), the humanworkers turn the board over by pushing it. Motivated by humanworkers’ actions in the sewage machine factory, we in thisresearch expect to perform a similar task using collaborativerobots that operate crane pulley blocks.We develop a planner that optimizes the action sequencesand plans the motion for a collaborative dual-arm robot tooperate crane pulley blocks and lift up and tumble heavy plate-shaped objects with the crane pulley blocks’ help. We assumetarget plates are pre-attached to the crane cook. Depth visionis used to recognize the position and orientation of the cranerope and the plates. Based on the recognized rope and plate a r X i v : . [ c s . R O ] J a n PREPRINTS AT ARXIV, SUBMITTED TO A JOURNAL FOR REVIEW, 2020 poses, we optimize the initial and goal pulling poses for therope and plan the robot’s dual-arm motion that pulls the cranerope to lift up the plates. When the plates are lifted up toa satisfying pose, we compute the relations between contactwrenches and gravitational forces, and plan a pushing motionfor one of the robot arms to tumble over the plates.Our main contribution is twofold. First, we initially developa dual-arm robot that manipulates a plate much heavier thanthe maximum end-effector load by operating crane pulleyblocks. Second, we formulated the constraints in using thecrane pulley blocks and performed multiple optimizations tolet the robot autonomously determine the pulling arms, pullingactions, and the tumbling trajectories. We present the technicaldetails of the planner and several experiments and analysiscarried out using the dual-arm robot and different plates. Theinfluence of various parameters and optimization goals areinvestigated and compared in depth. The results show that theproposed planner is flexible and efficient.In the remaining part of this article, we first review relatedwork in Section II. Then, we present an overview of themethod in Section III. After that, we show the technical detailsin Section IV, with experiments and analysis carried out usinga dual-arm robot made by two Universal Robots UR3 arms andplates with different parameters in Section V. Conclusions andfuture work are presented in the last section.II. R
ELATED W ORK
We review the related work in the manipulation planningof heavy objects. They are divided into two categories. Thefirst category concentrates on firmly grasped or firmly at-tached objects. The goal is to keep balance or avoid breakingweak joints. For example, Harada et al. [6] used the ZeroMoment Point (ZMP) of the humanoid robot-object systemto compensate a humanoid robot’s motion, and thus enablethe humanoid robot to lift up an carry a heavy object stably.Urakubo et al. [7] studied lifting up a heavy object with smalljoint torques. They take advantage of singular configurationsto avoid breaking the joints of a two-link robot. Berenson et al.[8] formulated the manipulation planning of heavy objects as aprobabilistic sampling problem while considering constrainedmanifolds. Yang et al. [9] developed a tele-operating systemconsidering mixed force and motion commands to avoidunexpected excessive force caused by massive payload. Zhu etal. [10] presented a planner for a wall climbing manipulator.The influence of self-weight was considered in determiningthe footholds for the suction modules. These studies assumethat a robot firmly grasps the heavy object, which could beundesirable as heavy objects are generally large. A roboticgripper may not be able to grasp them with force or formclosure. Also, even if a robot can grasp a part of a large,heavy object, it may remain challenging for the robot to pickup the object due to the fragile fingertip or fingerpad contacts.For this reason, non-prehensile manipulation is widely used toplan to manipulate heavy objects. It is the second category ofstudies we would like to review.Non-prehensile manipulation means manipulating an objectwithout firmly grasping it. The representative non-prehensile manipulation primitives include pushing, sliding, tumbling,pivoting, lifting, caging, etc. Pushing was extensively studiedby Mason [11], and is currently widely used to manipulate theobject. When a robot pushes an object, one needs to considerboth the trajectories of the object and robot itself. Lynch andMason [12], and more recently Zhou and Mason [13] analyzedthis control and planning problem and implemented stablepushing. Recent progress in pushing include Zhou et al. [14]and Yu et al. [15], which use probabilistic models to inferobject states. Song et al. [16] proposed a nested approachto manipulate multiple objects together using pushing andlearning. With the help of pushing, a robot can manipulateobjects that cannot be directly grasped and lifted. For example,Murooka et al. [2] presented a humanoid robot that pushedheavy objects by using whole-body contacts. Topping et al.[17] modeled and planned a small quadruped robot’s motionto open large doors.Sliding is similar to pushing, but instead of exerting forcesideways, sliding assumes pressing against the frictional sur-face. It also enables a small robot to manipulate large andheavy objects. Zhang et al. [18] presented a dynamic modelto plan the motion for a legged robot perform various slidingtasks like driving, inch worming, scooting, etc. Hang et al.[19] developed a pregrasp policy than slides thin objects tothe corner of a table for easier pick-up.Tumbling means rotating an object while pressing it againsta surface. Bai et al.[20] analyzed tumbling an object using amulti-fingered robotic hand. The motion is induced by a tiltedpalm and gravity. Fingers were used to protect the tumblingfrom overshooting. Cabezas et al. [21] presented a tumblingplanner that accepts a given trajectory of rotation and computesthe quasi-dynamic contacts. Correa et al. [22] and Cheng etal. [23] respectively developed new usage of suction cups byconsidering using them as a tip for tumbling.Rolling is a variation of tumbling, where the manipulatedobject is rotated continuously along a surface [24][25].Pivoting is a method that moves an object by leaving theobject alternatively supported by corner points as if the objectis walking on them. It is an extended version of tumbling.Aiyama et al. [26] seminally suggested the idea of roboticpivoting. Yoshida et al.[1] used a humanoid robot to pivot andmove a heavy box.Besides the primitives, researchers also employed high-levelgraph search to plan composite non-prehensile manipulation.For example, Maeda et al.[27] proposed using a manipulation-feasibility graph to host the contact states [28] of an objectand plan multi-finger graspless manipulation by searching thegraph. Lee et al. [29] proposed a hierarchical method thatused a contact-state graph in lower layers to identify object-environment contacts, and determined robot contact sequencesand maneuverings in a higher layer.Compared with the above non-prehensile manipulation, ourdifference is as follows. Firstly, we do not stick to a robotitself. Our non-prehensile manipulation is performed by usingcrane pulley blocks to increase the duty of robots. Similarideas could be found in a most up-to-date publication thatuse a collaborative robot to operate a manual pallet jack [30].We plan both the dual-arm robot action sequences and motion
AYAKAWA et al. : A DUAL-ARM ROBOT THAT AUTONOMOUSLY LIFTS UP AND TUMBLES HEAVY PLATES USING CRANE PULLEY BLOCKS 3 details to pull and return the crane pulley blocks’ rope andease non-prehensile flipping. Second, we plan a robot pushingmotion to turn over heavy plates. We study a quasi-staticprediction of a single contact, and use it to generate thetrajectory to tumble plates.III. O
VERVIEW OF THE P ROPOSED M ETHOD
This section presents an overview of our proposed method.We base our discussion on the hardware set up shown in Fig.2.The proposed method is not limited to this hardware setup,but we use it as an example to let our readers have a solidconception of the working scenario.Fig. 2: An exemplary hardware setup. The goal is to optimizethe robot action sequences and plan the robot motion that pullsup the heavy plate using the crane pulley blocks, and tumblesover the plate at a satisfying pose.The proposed method is divided into two phases. The firstphase is a rope pulling phase. In the beginning this phase,the method detects a rope’s position using point clouds ob-tained from a depth sensor. Many well-developed algorithms,for example, RANSAC [31] and ICP [32], can be used toperform the detection. The detected rope, together with thepre-annotated grasp poses that grasp a section of the rope,is used to examine which robot arm to use and optimize theinitial and goal rope-pulling poses. The two arms are examinedsequentially, but they do not necessarily move one by one. Theactuation of arms is determined by a quality function designedto maximize the pulling distance, minimize the pulling force,and enlarge grasping flexibility so that the plate object can bequickly and safely lifted up. An arm motion will be plannedand executed to pull the crane rope if an optimal goal isfound. After pulling the rope, the method performs a secondvisual detection to find the plate’s pose. It determines if aplate is at a satisfying pose for tumbling. If the pose is notsatisfying, the routine returns to the rope detection and pullingoptimization to pull down the rope continuously. Or else,the method switches to a tumbling phase that optimize thetrajectory of a robot arm that flips the plate using sliding-push. The tumbling is also monitored by vision to determine if theplate is well flipped or not. Once the plate’s pose is consideredto be reversed, the robot uses its two arms to return the cranerope and lower the plate down to the table.The detailed algorithms related to the two phases will bepresented in detail in Section IV and V, respectively. Theirperformance will be examined and analyzed in Section V.IV. P
ULLING THE R OPE U SING O PTIMIZED P OSES
In this section, we present the optimal rope-pulling planningalgorithms used in the rope pulling phase. The robot liftsup a plate by repeatedly carrying out the planned rope-pulling motion. As shown in Fig.3, the planning is performedrepeatedly in closed loops. Inside each loop, the robot uses aquality function to determine the best initial and goal pullingposes, and plans and executes the planned pulling actions untilthe plate is lifted up to a satisfying angle. If the planning orexecution in a loop fails, the planner will invalidate the failedgrasping point or pulling pose and try another candidate. Therobot autonomously switches between single-arm or dual-armactions following the determined goals.Fig. 3: Workflow of the optimal rope-pulling planning algo-rithms. The planning is performed repeatedly in closed loops,as denoted by the red arrows and green arrows. Inside eachloop, the robot uses a quality function to determine the bestinitial and goal pulling poses, and plans and executes theplanned actions until a plate is lifted over a threshold angle.The components of Fig.3 will be presented in detail in theremaining part of this section. Frame box 1) will be presentedin subsection A. Frame box 2) will be presented in subsectionB. Frame boxes 3)-5) will be presented in subsection C.The closed-loop arrows (the red arrows and the green arrowsrespectively form closed loops) and the switches of arms willbe presented in subsections D. Frame box 6) will be presentedin subsection E.
PREPRINTS AT ARXIV, SUBMITTED TO A JOURNAL FOR REVIEW, 2020
Fig. 4: (a) Searching for the initial grasp point and pulling poses considering cylinder elements and pre-annotated grasp poses.The lower part of the figure shows the cylindrical modeling of the rope and the pre-annotated grasps for a cylinder element.(b, c, d) Sampling and determining an optimal init-goal pair considering the pulling distance computed using l i , the pullingload computed using θ i , and the chance of successful motion planning computed by reasoning shared grasps. The randomgreen dots are the sampled goals. The two clusters of hands in (d) show the grasping poses at the initial and goal points. Thered and green ones are the shared grasping poses at both the initial and goal points. The blue ones in the upper cluster are notaccessible at the goal. In the shared grasping poses, the red ones are IK-infeasible or collided. The green ones are the finallydetermined candidate grasping poses. Their related arm pulling poses are also rendered in green color. A. Searching for the Initial Grasping Point and Pulling Poses
The initial grasping point and initial pulling poses for therope-pulling motion are determined considering the rope’spoint cloud obtained inside each loop. We use a series ofconnected cylinders to model a detected rope point cloud,as shown in Fig.4(a). The cylinder elements have the samesize. Grasp poses are pre-annotated for a cylinder element. Todetermine initial pulling poses, we scan the cylinder elementsof a detected rope from its top-most position and select thefirst one that is not invalidated by previous loops as the initialgrasping point. Then, the pose of an arm is computed bysolving Inverse Kinematics (IK) considering the pre-annotatedgrasp poses at the selected point. After that, the collision-free ones of the solved arm poses are saved as the initialpulling poses. The green rendering in the upper part ofFig.4(a) exemplifies a determined initial pulling pose. The redrendering shows an obsoleted grasping pose as the arm elbowat this configuration collides with the robot body.
B. Determine the Optimal Init-Goal Pair
After the initial grasping point and pulling poses are de-cided, the planner continues to decide goal points and thepulling poses at them to form init-goal pairs. The goal pointsare selected from a set of probabilistically sampled pointsin a robot arm’s workspace, as shown in Fig.4(b-d). Sincethe two arms have different workspaces, their goal points aresampled differently. The most effective point in the sampledset is selected to form the initial-goal pair, where effectivenessis evaluated considering three aspects: The motion distancebetween the init-goal pair; The load that a pulling arm bearswhen moving between the init-goal pair; The number of sharedgrasp poses between the init-goal pair. The three aspects are re-spectively quantified as quality parameters f length , f load , and f grasps . The values of these quality parameters are normalizedto (0, 1). Their details are as follows. f length : The f length quality assesses the motion distancebetween the init-goal pair. Its goal is to enlarge the length ofeach pulling motion. f length is computed using equation (1). f length = ( l i − l min ) / ( l max − l min ) . (1) Here, l i is the length between the upper pin point of the pulleyblocks and a sampled goal point. It is graphically explainedin Fig.4(b). l max and l min are respectively the shortest andlargest of all l i . The denominator ( l max − l min ) normalizesthe values. f length reaches to 1 when the furthest goal sampleis selected. f load : The f load quality assesses the load that a pullingarm bears when moving between the init-goal pair. Its goalis to reduce the load of a pulling arm. The f load value iscomputed using the tilting angle of the stretched rope. A FreeBody Diagram (FBD) analysis shows that a larger tilting anglerequires a robot arm to afford larger forces. Thus we use thecosine value of the angle to approximate f load : f load = cos θ i (2)Fig.4(d) graphically explains θ i . f load reaches 1 when the ropeis pulled vertically. The robot arm bears the smallest resistantforce at this extreme. f grasps : The f grasps quality aims to improve thechances of successful motion planning. It is computed usingthe number shared grasp poses at the initial grasping point andgoal point as follows: f grasps = n goal /n init . (3)A shared grasp pose is defined as the grasp poses identicalin a cylinder element’s local coordinate frame at both theinitial grasping point and the goal point. Shared grasps arecomputed by reasoning the intersection of all available (IK-feasible and collision-free) grasp poses at the init-goal pair. n init in the equation indicates the number of available graspposes at the initial grasping point. n goal indicates the numberof grasp poses at the goal point that shares the same localtransformation as those in n init . A large f grasps meansmore candidate initial and goal pulling poses for followingconfiguration-space motion planning, which implies a higherchance of getting a feasible motion trajectory.The most effective goal point is determined considering thethree quality parameters using the following quality function. Q = ω T f (4) AYAKAWA et al. : A DUAL-ARM ROBOT THAT AUTONOMOUSLY LIFTS UP AND TUMBLES HEAVY PLATES USING CRANE PULLEY BLOCKS 5
Here, ω =[ ω length , ω load , ω grasps ] is the weighting coefficient. f =[ f length , f load , f grasps ] indicates the three quality parame-ters. The quality function helps to find a goal point that leadsto large pulling distances, smaller pulling forces, and highchance of finding successful pulling motion. The weightingcoefficient can be tuned following application requirements.Their influences on final performance will be analyzed in theexperimental section.For all sampled goal points, we compute their Q andchoose the one with the largest Q value as the optimalgoal. The initial grasping point and the optimal goal pointtogether form an init-goal point pair. The goal arm pullingposes are determined by considering the collision-free InverseKinematics (IK) solutions of the shared grasp poses at theinit-goal pair. The two clusters of hands in Fig.4(d) show thegrasping poses at the initial and goal points. The blue ones inthe upper cluster indicate the unshared grasping poses. Theyare inaccessible at the goal. The ones with the other colorsindicate the shared grasping poses. In the shared graspingposes, the red ones are IK-infeasible or collided. The greenones are the finally determined candidates. Their related armpulling poses are rendered for better observation. C. Planning the Rope-Pulling Motion
After the initial and goal pulling poses are determined, therobot generates a motion to pull a rope from the initial pose tothe goal pose. The motion includes two sections. In the firstsection, the pulling arm moves from its current pose to theinitial pulling pose. The section is planned using Bi-directionalRapidly-exploring Random Trees (Bi-RRT) [8]. In the secondsection, the pulling arm pulls the rope by moving from theinitial pulling pose to the goal pulling pose. The section isplanned as a linear trajectory using Quadratic Programming(QP) [33][34]. If the planning succeeds, the robot will executethe successfully planned motion to pull up the plate. Or else,the planner will continue to try another init-goal pair. Theplanning and execution routine is performed repeatedly untilthe plate’s tilting angle exceeds a threshold. We use a thresholdbecause we need to ensure that the plate is not lifted toomuch and separated from the table. The plate continuouslycontacts with the table until the end of the rope-pulling phase,so that a robot arm could start tumbling manipulation. Beforethe execution in each loop, the plate’s post-execution tiltingangle is predicted to prevent it from exceeding the threshold.The post-execution angle prediction is performed using thefollowing equation. ˜ α i +1 = α i + ( α i − α i − ) × d i d i − , ( i ≥ (5)Here, α i − and d i are respectively the previous plate’s tiltingangle and the length of the previous rope-pulling motion. α i and d i are the current plate’s tilting angle and the length ofthe current rope-pulling motion. The plate’s tilting angle afterexecuting the current rope-pulling motion is estimated as ˜ α i +1 .The various symbols are graphically explained in Fig.5. Theprediction is based on the proportional relationship betweenthe length of the pulled rope and the change of the anglein a previous execution. It is decoupled from a specific plate and provides an upper-bound estimation for the post-executionangle. Fig. 5: Predictinga plate’s next tilt-ing angle ˜ α i +1 us-ing the current ti-tling angle α i andprevious tilting an-gle α i − .If the predicted angle exceeds the threshold, the length of thenext pulling motion will be adjusted to avoid over-lifting. Thefollowing equation shows the adjustment. The plate’s tiltingangle after finishing the rope-pulling phase approximates tothe threshold angle α thld . ˆ d i = d i − × α thld − α i α i − α i − (6) D. Determining the Action Arms and Sequences
As mentioned in C , the planning and execution loop isperformed repeatedly until the plate’s tilting angle exceedsa threshold. The two arms share the repetition sequentially –The planner optimizes init-goal pairs and plans and executesa pulling-motion for the two arms one by one. It switchesto a second arm either after pulling the cable or without anypulling action. Particularly, if no init-goal pair is found or nosuccessful motion is planned, the planner switches to the nextarm without pulling, exhibiting autonomy in determining theaction arms and sequences.The arrows in Fig.3 show the mentioned switches. The greenarrows indicate the flow with pulling executions. The currentarm along the flow is actuated to pull the cable. It will switchto the other arm after execution. The red arrows indicate theflow without pulling. During the planning, a failure may becaused by different reasons like no available initial pose, noavailable init-goal pair, failed to find a motion for an init-goalpair, etc. If the reason is a collision between the lifted plate andthe other arm, the planner will try moving the other arm awayand continue the currently planned results (as shown by “Yes ”in Fig.3). Or else, the planner will jump over the currentinit-goal pair and perform replanning by iterating to othercandidates (as shown by the “No” condition after “ Succ. ”and the “Yes ” condition after “ P lateColl. ” in the figure).If none of the sampled goals form a feasible init-goal pair,the current arm will not perform the pulling action. Instead,it simply re-grips by opening the gripper, moving to the initpulling pose, and closing the gripper. The planner switchesto plan for the other arm after the re-gripping. A final failureis reported when all cylinder elements on a detected rope aretraversed, and none of them lead to a feasible initial pullingpose.Following the green and red workflows, the action armsand sequences are determined autonomously. In an extremecase, a robot may use a single arm to pull while never findfeasible goals for the other arm. The robot may also use thetwo arms one by one in another extreme case where both arms
PREPRINTS AT ARXIV, SUBMITTED TO A JOURNAL FOR REVIEW, 2020 find feasible init-goal pairs. More moderate cases are that therobot sometimes uses a left arm and sometimes uses a rightarm, depending on the amount of feasible init-goal pairs foundduring the repeated planning.Fig.6 shows two examples. In Fig.6(a), the robot is pullingup a small plate. The robot finished a right-arm executionin (a.1) and is checking the collisions of the shared left-arm grasping poses. The top-most cylinder element on thedetected rope is chosen as the init point. The rendered leftarms are the shared IK-feasible poses. There are both collidedand collision-free ones. The collided poses are shown in red,and the collision-free poses are shown in green. In (a.2), therobot plans a linear motion to move the left arm from an initialpulling pose to a goal pulling pose. Fig.6(b) shows the caseof larger plate. All shared IK-feasible grasp poses of the leftarm collide with the plate in (b.1). The planner reaches to afinal failure and switches to the right arm without triggeringexecution. In (b.2), the right arm finds a new init-goal pair andplans a linear motion to move the right arm from an initialpulling pose to a goal pulling pose.Fig. 6: Two examples of switching the action arms. The goalis to lift the board from the yellow pose to the green pose.(a) The robot successfully finds an init-goal pair for its leftarm and plans a pulling motion. (b) The robot fails to find agoal for its left arm because of the bulky plate’s obstruction.It switches to the right arm without execution.
E. Include Re-Recognition in the Loop
The plate pose is re-recognized using a depth cameraafter each execution to acquire its real tilting angle. Likerecognizing a rope, RANSAC and ICP are used to extract theplate’s largest planar surface. The real tilting angle of the plateis computed considering the normal of the extracted planarsurface. It is then used to update the α i in equation (5) as wellas to predict the ˜ α i of the next rope-pulling action. Fig.7 showsan example. The point cloud of the scene in (a) is acquiredand shown in (b). The estimated planar surface normal and realtilting angle are illustrated together with the point cloud. Theoptimization, planning, execution, and recognition workflowsmentioned in this section’s subsections are repeated for thetwo arms sequentially until the plate is lifted up to a giventhreshold angle. Fig. 7: A plate’s real tilting angle is re-recognized using adepth camera after each execution. The algorithm extractsthe largest planar surface and computes the real tilting angleconsidering the normal of the extracted surface.V. T UMBLING THE P LATE U SING S LIDING -P USH
After lifting the plate to the threshold angle, one robotarm will tumble it over by pushing. Here, we assume thata plate always has edge contacts with the table. It will betumbled by rotating around the edge contacts. The rotationtrajectory of a plate is modeled as a time-variant function, andthe robot arm follows the trajectory by using sliding-push. Wedefine a sliding-push as a pushing policy that allows sliding onthe contact surface of a plate. Like the rope-pulling actions,the sliding-push motion is performed continuously until theplate is rotated beyond a threshold pose where the force andmoment reach equilibrium when the robot moves away fromthe pushing point.Besides the edge contacts, we also assume the followingconditions and constraints in planning the sliding-push: (1)All forces appear inside a vertical plane perpendicular to thecontact edges; (2) The plate and the environment are rigid;(3) The push between two nearby time instants is quasi-static;(4) The friction at the contact point follows the Coulombfriction model; (5) Plate-slipping on the table surface is notconsidered, but finger-sliding on the contact surface is allowed.Especially the fifth assumption is a strong constraint andleads to fewer solutions, but we keep it there to narrowdown the search space. A plate has three contacts duringpushing – Rope connection, robot fingertip contact, and tablecontact. The force at the connecting point with the rope isdifficult to control. The contact between the fingertip and theplate is assumed to be a point contact. During pushing, theplate may both slide on the table surface and rotate aroundthe contact edge, making it extremely difficult to performoptimization. To avoid these problems, we develop the fifthassumption to constrain a robot arm rotating a plate withoutallowing it to slide on the table surface. Pushing actions willbe carefully optimized considering the constraint about table-slipping. Meanwhile, we allow a pushing finger to slide on theplate’s contact surface to maintain high success rate.Based on these assumed constraints, a plate’s motion tra-jectory is determined once the contact edge is known. Thetumbling action is implemented by optimizing a robot arm’spushing trajectory while considering the plate’s motion trajec-tory. As the pushing arm tumbles a plate, the remaining robotarm holds or loosens the crane rope alternatively to avoid jerks.Take the plate pose at a time instant t i shown in Fig.8 forexample. We divide the pushing at this time instant into two AYAKAWA et al. : A DUAL-ARM ROBOT THAT AUTONOMOUSLY LIFTS UP AND TUMBLES HEAVY PLATES USING CRANE PULLEY BLOCKS 7 states s i and s (cid:48) i . The plate is pushed from a previous pose at t i − to the current pose in the first state. The crane rope isstraightened, and a tension force T appears along the rope. Inthe second state, the rope is loosened by the other robot arm,and the rope tension T disappears. The tumbling is performedacross a sequence of these states at different time instants inthe following way. s lsn −−→ s (cid:48) (cid:124) (cid:123)(cid:122) (cid:125) t hld −−→ s lsn −−→ s (cid:48) (cid:124) (cid:123)(cid:122) (cid:125) t hld −−→ . . . s n − lsn −−→ s (cid:48) n − (cid:124) (cid:123)(cid:122) (cid:125) t n − hld −−→ s n (cid:124)(cid:123)(cid:122)(cid:125) t n Between each s (cid:48) i − and s i , the other arm holds ( hld forabbreviation) the crane rope. Between each s i and s (cid:48) i , the cranerope is loosened ( lsn for abbreviation). We analyze the forcesat each time instant and determine the pushing points at eachof them by minimizing the balancing forces and reducing therope tension. The formal expressions for the minimization isas follows.min r k ( F T F ) + k ( F T F ) + k ( T T T ) (7a)s.t. T + G + (cid:88) j =0 F j = 0 (7b) r t × T + r g × G + (cid:88) j =0 r j × F j = 0 (7c) F T F ∈ (0 , (7d) f x f y ∈ (0 , µ ) (7e) acos ( F · r || F |||| r || ) ∈ ( atan µ , π − atan µ ) (7f) r ( t i ) T r ( t i ) ∈ (0 , l ) (7g) | r ( t i ) − r ( t i − ) | ≤ | v max | ( t i − t i − ) (7h) acos ( ( r ( t i +1 ) − r ( t i )) · ( r ( t i ) − r ( t i − )) || r ( t i +1 ) − r ( t i ) |||| r ( t i ) − r ( t i − ) || ) ≤ γ (7i)The meanings of the various variables are listed below.They are also graphically illustrated in Fig.8 for readers’convenience. F Force at the table contact point p in an s i state. An edgecontact is simplified into a point contact. F = [ f x , f y ]. F Force at the robot finger tip contact point p (pushingpoint) in an s i state. F = [ f x , f y ]. G Gravity. T Tension caused by a stretched crane rope. Only exist atan s i state. r t The vector pointing from the rotation center to the ropeconnecting point. r g The vector pointing from the rotation center to the plate’scenter of mass. r j The vector pointing from the rotation center to p i . µ The Coulomb friction coefficient at p . µ The Coulomb friction coefficient at p . v max The maximum speed of a pushing arm. ( t i ) If a symbol does not have this postscript, it alwaysdenotes a value at time instant t i . Or else, the symbol denotes a value from the time instant shown in theparenthesis. γ The maximum allowable angle between current and pre-vious pushing directions.Fig. 8: The rotational trajectory of a plate is modeled as atime-variant function across t , t , ..., t n . A pushing at a timeinstant t i is divided into two states s i and s (cid:48) i . The goal ofoptimization is to simultaneously minimize F , F , and T ,while considering a changing r = −−→ p p (sliding-push).The optimization goal show in equation (7a) is to minimizethe balancing force needed in the entire system. The ideabehind this optimization is that by minimizing the forceapplied to the entire system, the robot can push the platewith a minimum force and reduce the external forces fromthe rope and the desk. The smaller forces will also decreasethe risk of rotating in an unexpected direction (caused by non-vertical rope tension) or slipping on the table. The constraints(7b), (7c) balance the forces and torques at an s i state. Theconstraints (7d), (7e), (7f) add bounds to the force exertedby a robot and the friction cone at the contact points at the s i state. The constraint (7g) limits r to be on a contactsurface. The constraint (7h) ensures the contact points at twoconsecutive time instants are reachable. The last constraint(7i) smooths the change in the pushing direction. It preventsthe pushing direction at a time instant from getting largelydiverted from its precedence. Both constraints (7g) and (7i)are important to finding a practical pushing trajectory. Theirroles and difference are illustrated in Fig.9. Essentially, theconstraint (7g) limits the pushing distance between adjacentpushing points. The constraint (7i) limits the changes inpushing directions.Fig. 9: (a) Equation (7h) constrains the maximum distancebetween adjacent pushing positions. It makes the blue arrowsin the figure short. (b) Equation (7i) constrains the deviationof a subsequent pushing direction with its precedence. It helpsto avoid oscillation and make the trajectory smooth. PREPRINTS AT ARXIV, SUBMITTED TO A JOURNAL FOR REVIEW, 2020
Fig. 10: (a.1-2) The optimized trajectories experience slightchanges as the plate rotates onto a second contact edge.The change gets significant as the plate becomes thick. (b)Returning the rope using both arms to completely lower downa plate onto the table.The minimization is performed across all s , s (cid:48) , s , s (cid:48) , ...states to determine a sequence of optimal r , say an optimalsliding-pushing trajectory. Note that during the optimizationthe robot kinematic constraint is not considered. Instead of in-cluding it integrally as a constraint, we examine the kinematicconstraint lazily after a sequence is found. If the robot IK isnot solvable, we switch to the next best r until success or afailure is reported. With the IK constraints, the optimizationwill find both a sequence of pushing points and a sequence ofrobot joint trajectories that produce the pushing motion alongwith the pushing points. The joint trajectories will be executedby the target arm. The other arm plays the role of holdingand loosening the crane rope following the changes of states.As mentioned at the beginning of this section, the resultedmotion is a sliding-push. The contact points at different t i arenot necessarily the same in a plate’s local coordinate system.The pushing finger may slide on the contact surface overconsecutive t i during the tumbling.In addition, the contact edge between a plate and a tablesurface may change during the tumbling as a plate alwayshas a thickness. Our tumbling planner takes into accountthe changes of the contact edge and optimizes the pushingtrajectory considering the changing rotating axes. Fig.10(a.1-2) show the found pushing trajectories for two plates withdifferent thickness. The trajectories experience slight changesas the plate rotates onto a second contact edge.Finally, after the pushing robot arm finishes the wholeplanned trajectory, it works together with the other arm toreturn the rope and to further lower down the plate on thetable. Fig.10(b) illustrates the returning actions. They are alsoplanned online with visual rope detection. The plate’s realtilting angle is monitored during returning to make sure itis completely lowered onto the table. The difference is thereis no optimization, and the robot moves along a straight linepointing to the upper pulley block. The full task is reported tohave been completed after the plate reaches the table surface.VI. E XPERIMENTS AND A NALYSIS
The proposed method is implemented and examined usingthe robot system shown in Fig.2. The system includes twoUR3e robots with two Robotiq F85 two-finger grippers at eachrobot end flange. A Kinect V2 (Microsoft) sensor is used to acquire 3D point clouds. The computer used for planning andoptimization is a PC with Intel Core i9-9900K CPU, 32.0GBmemory, and GeForce GTX 1080Ti GPU. The programminglanguage for implementing the algorithms is Python.
A. Influence of the Goal Quality Function for Pulling
First, we analyze the influence of the quality functionpresented in equation (4), and examine the changes of theselected pulling goals under different weights. We perform theanalysis by setting one of the three weights to a higher valuewhile keeping the other two to zero and observe the plannedresults. Specifically, we compare ω = [ ω length , ω load , ω grasps ]= [1 , , T , [0 , , T , and [0 , , T and show the distributionof the selected goals for the left and right arms respectively. ω = [1 , , T : In this case, only f length influences therope-pulling motion. It drives the robot to select goal pointsthat lead to a large pulling distance. Fig.11(a) and (b) show anexample of a selected init-goal pose pair and a distribution ofthe selected goal points under the weight setting, respectively.The robot selects faraway points for both the left and rightarms to pull the rope with long distances.Fig. 11: Results when ω = [1 , , T : (a) An example of aselected init-goal pose pair. (b) A distribution of the selectedgoals under 15 times of simulation. ω = [0 , , T : In this case, only f load affects the pullingmotion. Fig.12 exemplifies an init-goal pair and a statisticaldistribution of the selected goal points. The robot tends toselect goals that make the stretched rope to have small tiltingangles to reduce the forces that an arm needs to bear.Fig. 12: Results when ω = [0 , , T : (a) An example of aselected init-goal pose pair. (b) Distribution of the selectedgoals after 15 times of simulation. ω = [0 , , T : In this case, only f grasps affects therope-pulling motion. This parameter is calculated based onthe number of grasping poses simultaneously available to theinitial and goal points of a pulling motion. Since the two AYAKAWA et al. : A DUAL-ARM ROBOT THAT AUTONOMOUSLY LIFTS UP AND TUMBLES HEAVY PLATES USING CRANE PULLEY BLOCKS 9 arms have higher manipulability near the body center, therobot has more feasible grasp poses when a goal point isnear the central line. The expectation is validated by Fig.13.In the upper two examples of this figure, the left and righthands pull the rope to points near the center. In the lower twoexamples, we intentionally included an obstacle at the centerfor comprehension. When there are obstacles, the number ofgrasp poses near the center is reduced. As a result, points awayfrom the center are selected as goals. The f grasps parameterallows the robot to select goals considering both kinematicand geometric constraints.Fig. 13: Results when ω = [0 , , T : (a.1) An exampleof a selected init-goal pose pair. (a.2) Distribution of theselected goals under 15 after times of simulation. (b.1-2) Theexemplary selected pair and sample distributions when anobstacle (the purple block) is intentionally placed in the robotworkspace. B. Influence of the Constraints and Parameters for TumblingOptimization
Second, we study the influence of different constraints andparameter settings on the optimized tumbling motion. Weare especially interested in: (1) Necessity of the trajectoryconstraints (7h), (7i); (2) Different values of v max and γ ; (3)Different friction coefficients, center of mass (com) positions,and hooking positions. For each of the different constraints andvalues, we run our algorithms using plates of different sizes(40 mm × mm , 150 mm × mm , 44 mm × mm ) andcompare the optimized results. The magnitudes of the sizesare coherent with the ones used in the real-world experiments(Table I).
1) Necessity of the trajectory constraints (7h) and (7i) : We study the change of the trajectories under constraints (7h)and (7i) and examine their necessity. The results are shown inFig.14. Each column of the figure uses a plate of a differentsize. The four rows are respectively: (a.1-a.3) Both (7h) and(7i) are considered; (b.1-b.3) Only (7h) is considered; (c.1-c.3) Only (7i) is considered; (d.4-d.3) Neither (7h) nor (7i)is considered. The results imply that both (7h) and (7i) arenecessary for obtaining a smooth and stable pushing trajectory. When constraint (7i) is removed, the trajectory gets oscillated.The oscillation can be observed by comparing the trajectoriesin sub-figure (b.2) with that of (a.2). When constraint (7h)is removed, the distance between adjacent pushing pointsbecomes large. The large distance can be easily observed bycomparing the second half of the trajectory in (c.2) with thatof (a.2). When both (7h) and (7i) are omitted, the optimizedtrajectories involve jitters and jerks and become unsuitable forrobotic execution. Note that in getting these results, we set theparameters v max = mm / s , and γ = ◦ when they are used.Fig. 14: Necessity of the trajectory constraints (7h) and (7i).Each column of the figure uses a plate with a different size(40 mm × mm , 150 mm × mm , 44 mm × mm ). Thefour rows are respectively: (a.1-a.3) Both (7h) and (7i) areconsidered; (b.1-b.3) Only (7h) is considered; (c.1-c.3) Only(7i) is considered; (d.4-d.3) Neither (7h) nor (7i) is considered.
2) Different values of v max and γ : In this part, we in-vestigate the influence of the parameters v max and γ inthe trajectory constraints on the optimization. The resultedtrajectories using different v max and γ values are shown inFig.15. Like Fig.14, each column of the figure uses a platewith a different size. For the results in the upper two rows,the γ parameter in constraint (7i) is changed, while v max isfixed. For the results in the lower two rows, the value of v max in constraint (7h) is changed, and (7i) is fixed.For Fig.15(a.1) and (b.1), we set v max = mm/s and mm/s respectively, and fixed γ = ◦ . No significant Fig. 15: Influence of v max and γ values on the optimized tra-jectories. (a.1-a.3) and (b.1-b.3): The γ parameter in constraint(7i) is changed, while v max is fixed; (c.1-c.3) and (d.1-d.3):The value of v max in constraint (7h) is changed and (7i) isfixed.change is observed in this case as the constraint (7i) playsa master role. Likewise, for (a.3) and (b.3), we set v max = mm/s and mm/s , and fixed γ = ◦ . There is nosignificant change observed. For (a.2) and (b.2), we set v max = mm/s and mm/s , and fixed γ = ◦ . In this case, thedistances between adjacent pushing points become larger. Theobservation shows that the constraint (7h) is effective.Specifically, for Fig.15(c.1) and (d.1), we set γ = ◦ and ◦ respectively, and fixed v max = mm/s . By comparingthem, we can observe that as the value of γ increases, thetrajectory biased downward. For Fig.15(c.2) and (d.2), we set γ = ◦ and ◦ respectively, and fixed v max = mm/s .By comparing them, we can observe that as the value of γ increases, the degree of oscillation became stronger. ForFig.15(c.3) and (d.3), we set γ = ◦ and ◦ respectively,and fixed v max = mm/s . The results show that the down-ward bias becomes slightly larger in the second half of thetrajectory (although no significant difference is observable).The observation indicates that the constraint (7i) is effective.
3) Different friction coefficients, center of mass (com) po-sitions, and hooking positions:
In this section, we investigatethe changes in trajectory when the friction coefficient at the Fig. 16: Influence of different friction coefficient ( µ ), centerof mass (com) position ( r g ), and hooking position ( r h ) on theoptimized trajectories. (a.1-a.3) µ = 0.05; (b.1-b.3) µ = 0.6;(c.1-c.3) r g is changed to the r g shown in Fig.17; (d.1-d.3) r h is changed to the r h shown in Fig.17.pushing point ( µ ), center of mass (com) position ( r g ), orhooking position ( r h ) vary. The results are shown in Fig.16.Each column of the figure uses a plate of a different size.For Fig.16(a.1), (a.2), and (a.3), µ is set to 0.05. Whenthe plates are thin, such as (a.1) and (a.3), the optimizedtrajectories bias downward in the second half. In contrast,when plates are thick, like the one shown in (a.2), theoptimized trajectory biases upward. For Figs.16 (b.1), (b.2),and (b.3), µ is changed to 0.6. Compared with the resultsof 0.05, the optimized trajectories kept neutral in (b.1) and(b.3) but biased downward in (b.2). The results show that µ significantly influences the choices of pushing points in thesecond half of the trajectory.Next, we investigate the influences of r g and r h . Weespecially consider two choices for them, as shown by r g , r g , r h , and r h in Fig.17. r g and r g are the geometriccenter and 1/4th of the geometric center of the plate rectangle. r h and r h are the top-left and top-right corners of the platerectangle. All previous trajectories were obtained considering r g and r h . In this part, we change them to the otherpositions for comparison. Fig.16(c.1), (c.2), and (c.3) showthe trajectories with respect to r g and r h . The results show AYAKAWA et al. : A DUAL-ARM ROBOT THAT AUTONOMOUSLY LIFTS UP AND TUMBLES HEAVY PLATES USING CRANE PULLEY BLOCKS 11
Fig. 17: We study the influence ofthe center of mass (com) position( r g ) and hooking position ( r h )on the optimized trajectories byvarying each of them to two dif-ferent positions. These positionsare denoted by r g , r g , r h , and r h in the figure.that when the com is shifted to the bottom of the plate, themaximum duty of a robot hand gets smaller and the handadjusts itself in wider range. Fig.16(d.,1), (d.2), and (d.3) showthe trajectories with respect to r g and r h . For all of them,the starting positions get lower. This may be because the robothand bears a smaller force when pushing starts from a pointfar from hooking position. C. Results of Various Plates
Third, we examine the performance of the proposed methodusing three plates shown in Fig.18 – An acrylic board, astainless box, and a plywood board. The parameters of theseplates are shown in Table I. We assume a plate initially lieson the table in front of the robot and the crane hook is pre-connected.Fig. 18: Three different plates used in the experiments. (a)An Acrylic Board. (b) A Stainless Box. (c) A Plywood Plate.Their various parameters are shown in Table I.TABLE I: Parameters of the Plates Used in Experiments
Plate Name h, l, w (mm) m (kg) r g µ µ Acrylic Board 300, 300, 40 4.0 Geom.Cent. 0.5 0.4Stainless Box 300, 400, 150 6.0 Geom.Cent. 0.4 0.1Plywood Board 500, 400, 44 6.4 Geom.Cent. 0.6 0.3 * l - length; w - width; h - height; m - mass. Fig.19 shows some snapshots of both the planning environ-ment and real-world execution results for the three plates. Inthese cases, the weight of the goal quality function is chosenas ω = [1 , , T . The robot managed to pull up the plates withsatisfying performance. Readers are encouraged to watch thesupplementary video submitted together with this manuscriptto better observe the optimized lifting up actions and tumblingtrajectories for the different plates. The most important points we are interested in for the real-world execution are the number of pulling actions, the pullingdistances, and the forces born by the pulling hands. We showthem in detail in Tables II-IV.First, in Table II, we compare the execution results of ω = [1 , , T with other three extreme candidates ω = [1 , , T , ω = [0 , , T , and ω = [0 , , T . The results indicate that whenthe weight is chosen as ω = [1 , , T , the robot works with asmaller number of actions and relatively large average pullingdistances. Meanwhile, the hands bear moderate pulling forces.When the weight is chosen to be ω = [1 , , T , the rope’spulling length becomes dominantly large, and the robot handsbear more forces. When the weight is chosen as ω = [0 , , T ,the forces born by the hands are highly suppressed, but thenumber of actions increase. When the weight is chosen as ω = [0 , , T , there is no significant influence on the lengthand force values. However, without the parameter f manip , therobot and obstacles’ collisions cannot be taken into account.Table III further shows the details of pulling lengths andforces for each pulling action when w is chosen as [ , , ] T .The robot alternatively used its left and right arms to pull upthe acrylic board and the stainless box. For the plywood board,the robot alternatively used the two arms in the beginning butswitched to right-alone actions after step 6. The reason is theplywood board is a large plate. It blocked the right arm actionafter being to lifted to a high pose.Table IV shows the detailed average time costs for theexecutions of lifting and tumbling each plate using ω = [1 , , T . The most time-consuming part of the proposedplanner is optimizing the init-goal pair, including samplinggoal points, evaluating goal grasps, and reasoning shared initialand goal grasp poses. They have to be performed on allsampled points to get the defined qualities. The QP solverand RRT planner used to generate the movement and pullingactions also take some time to search and optimize.VII. C ONCLUSIONS
This paper proposed an optimization and planning methodfor a dual-arm robot to lift up and tumble heavy plates usingcrane pulley blocks. The optimal pulling actions for the pulleybocks were achieved by maximizing or minimizing the lengthof a pulling motion, the load born by the pulling hand, andthe chances of finding a success motion. Visual detection wasincluded in each optimization and execution loop to update thestates of the plate. The optimal tumbling was implemented byconsidering the force and moment applied to the plate. Anoptimal sliding-push trajectory was planned by minimizingthe forces needed to maintain equilibrium. After tumbling,the plate was lowered down onto the table to completelyfinish a task. The action sequences and motion details of bothpulling and tumbling were autonomously decided followingrespective optimizations. Experiments and analysis showedthat the optimizations responded well to changing plate sizes,weights, materials, etc. The robot was able to flexibly andefficiently adapt its action sequences and motion details todifferent scenarios.The focus of this manuscript is the optimization and plan-ning aspect. We did not consider the rope and assumed it to
Fig. 19: Snapshots of the robotic executions for the three plates ( ω = [1 , , T ; Watch our supplementary video for details).TABLE II: Comparison of the Real-World Pulling Actions Under Different Weights w = [1,1,1] T w = [1,0,0] T Plate Name
Acrylic Board L2, R3 (367.68, 430.06) (3.98, 6.42) L2, R3 (448.50, 456.41) (6.64, 7.48)Stainless Box L2, R3 (294.99, 429.37) (5.70, 6.51) L2, R3 (350.41, 422.39) (5.83, 7.17)Polywood Board L3, R6 (332.65, 388.62) (6.07, 8.12) L3, R6 (411.65, 437.13) (8.21, 8.12)Total Average 3.17 373.90 6.13 3.17 421.08 7.24 w =[0,1,0] T w =[0,0,1] T Plate Name
Acrylic Board L3, R3 (348.71, 340.69) (4.93, 3.86) L4, R5 (283.86, 257.14) (5.59, 5.96)Stainless Box L3, R3 (236.63, 350.60) (5.64, 5.94) L4, R4 (288.68, 297.82) (7.17, 5.92)Polywood Board L3, R7 (334.52, 327.46) (5.99, 5.86) L4, R9 (297.88, 296.20) (6.62, 7.45)Total Average 3.67 323.10 5.37 5.00 286.93 6.45
TABLE III: Detailed Pulling Length and Forces of Each Pulling Action Using w =[ , , ] T Plate Name Item mm ) 443.64 377.75 471.32 357.61 375.23 - - - -Force ( N ) 8.48 4.3 5.58 3.65 5.72 - - - -Time ( s ) 7.57 6.09 4.02 5.94 4.36 - - - -Stainless Box Actn. Arm Rgt Lft Rgt Lft Rgt - - - -Distance ( mm ) 449.99 328.19 475.72 261.78 362.41 - - - -Force ( N ) 7.80 5.45 5.71 5.95 6.01 - - - -Time ( s ) 6.41 8.45 4.08 5.87 3.96 - - - -Polywood Board Actn. Arm Rgt Lft Rgt Lft Rgt Lft Rgt Rgt RgtDistance ( mm ) 455.69 367.77 449.4 343.95 425.74 286.24 394.68 394.94 211.28Force ( N ) 10.79 6.08 8.6 5.88 5.49 6.26 8.21 7.22 9.41Time ( s ) 8.55 5.39 4.00 5.99 4.47 6.36 3.99 4.15 3.85 TABLE IV: Time Costs for Each Part of A Pulling Action
Acrylic Stainless Polywood
Items
Rgt Lft Rgt Lft Rgt LftInitial Grasps ( s ) 0.16 0.59 0.15 0.58 0.18 0.55Init-Goal Pairs ( s ) 3.02 4.31 3.13 4.28 3.13 4.38QP + RRT ( s ) 1.41 1.76 1.43 1.78 1.48 1.69 be pre-attached to the plates. Autonomously manipulating andattaching a crane rope using the robots and peripherals willbe our future work. R EFERENCES[1] E. Yoshida, P. Blazevic, and V. Hugel, “Pivoting manipulation of alarge object: A study of application using humanoid platform,” in
IEEEInternational Conference on Robotics and Automation (ICRA) , 2005, pp.1040–1045.
AYAKAWA et al. : A DUAL-ARM ROBOT THAT AUTONOMOUSLY LIFTS UP AND TUMBLES HEAVY PLATES USING CRANE PULLEY BLOCKS 13 [2] M. Murooka, S. Nozawa, Y. Kakiuchi, K. Okada, and M. Inaba, “Whole-body pushing manipulation with contact posture planning of large andheavy object for humanoid robot,” in
IEEE International Conference onRobotics and Automation (ICRA) , 2015, pp. 5682–5689.[3] F. Ohashi, K. Kaminishi, J. D. F. Heredia, H. Kato, T. Ogata, T. Hara,and J. Ota, “Realization of heavy object transportation by mobile robotsusing handcarts and outrigger,”
Robomech Journal , vol. 3, no. 1, p. 27,2016.[4] Y. Hou, Z. Jia, and M. T. Mason, “Fast planning for 3d any-pose-reorienting using pivoting,” in
IEEE International Conference onRobotics and Automation (ICRA) , 2018.[5] J. A. Haustein, S. Cruciani, R. Asif, K. Hang, and D. Kragic, “Placingobjects with prior in-hand manipulation using dexterous manipulationgraphs,” in
IEEE-RAS International Conference on Humanoid Robots(Humanoids) , 2019, pp. 453–460.[6] K. Harada, S. Kajita, H. Saito, M. Morisawa, F. Kanehiro, K. Fujiwara,K. Kaneko, and H. Hirukawa, “A humanoid robot carrying a heavyobject,” in
IEEE International Conference on Robotics and Automation(ICRA) , 2005, pp. 1712–1717.[7] T. Urakubo, H. Yoshioka, T. Mashimo, and X. Wan, “Experimental studyon efficient use of singular configuration in pulling heavy objects withtwo-link robot arm,” in
IEEE International Conference on Robotics andAutomation (ICRA) , 2014, pp. 4582–4587.[8] D. Berenson, S. S. Srinivasa, D. Ferguson, and J. J. Kuffner, “Ma-nipulation planning on constraint manifolds,” in
IEEE InternationalConference on Robotics and Automation (ICRA) , 2009, pp. 625–632.[9] J. Yang, A. Konno, S. Abiko, and M. Uchiyama, “Hardware-in-the-loop simulation of massive-payload manipulation on orbit,”
RobomechJournal , vol. 5, no. 1, p. 19, 2018.[10] H. Zhu, J. Lu, S. Gu, S. Wei, and Y. Guan, “Planning 3d collision-freeoptimized climbing path for biped wall-climbing robots,”
IEEE/ASMETransactions on Mechatronics , pp. 1–1, 2020.[11] M. T. Mason, “Mechanics and planning of manipulator pushing oper-ations,”
The International Journal of Robotics Research , vol. 5, no. 3,pp. 53–71, 1986.[12] K. M. Lynch and M. T. Mason, “Stable pushing: Mechanics, controlla-bility, and planning,”
The International Journal of Robotics Research ,vol. 15, no. 6, pp. 533–556, 1996.[13] J. Zhou, Y. Hou, and M. T. Mason, “Pushing revisited: Differential flat-ness, trajectory planning, and stabilization,”
The International Journalof Robotics Research , vol. 38, no. 12-13, pp. 1477–1489, 2019.[14] J. Zhou, J. A. Bagnell, and M. T. Mason, “A fast stochastic contact modelfor planar pushing and grasping: Theory and experimental validation,”in
Robotics: Science and Systems (RSS) , 2017.[15] K.-T. Yu, M. Bauza, N. Fazeli, and A. Rodriguez, “More than amillion ways to be pushed. a high-fidelity experimental dataset of planarpushing,” in
IEEE/RSJ International Conference on Intelligent Robotsand Systems (IROS) , 2016, pp. 30–37.[16] C. Song and A. Boularias, “Object rearrangement with nested nonpre-hensile manipulation actions,” in
IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS) , 2019, pp. 6578–6585.[17] T. T. Topping, G. Kenneally, and D. E. Koditschek, “Quasi-static anddynamic mismatch for door opening and stair climbing with a leggedrobot,” in
IEEE International Conference on Robotics and Automation(ICRA) , 2017, pp. 1080–1087.[18] G. Zhang, S. Ma, Y. Shen, and Y. Li, “A motion planning approach fornonprehensile manipulation and locomotion tasks of a legged robot,”
IEEE Transactions on Robotics , vol. 36, no. 3, pp. 855–874, 2020.[19] K. Hang, A. S. Morgan, and A. M. Dollar, “Pre-grasp sliding manipula-tion of thin objects using soft, compliant, or underactuated hands,”
IEEERobotics and Automation Letters , vol. 4, no. 2, pp. 662–669, 2019.[20] Y. Bai and C. K. Liu, “Dexterous manipulation using both palm andfingers,” in
IEEE International Conference on Robotics and Automation(ICRA) , 2014, pp. 1560–1565.[21] B. Aceituno-Cabezas and A. Rodriguez, “A global quasi-dynamic modelfor contact-trajectory optimization in manipulation,” in
Robotics: Sci-ence and Systems (RSS) , 2020.[22] C. Correa, J. Mahler, M. Danielczuk, and K. Goldberg, “Robust topplingfor vacuum suction grasping,” in
IEEE International Conference onAutomation Science and Engineering (CASE) , 2019, pp. 1421–1428.[23] X. Cheng, Y. Hou, and M. T. Mason, “Manipulation with suctioncups using external contacts,” in
International Symposium on RoboticsResearch (ISRR) , 2019.[24] Y. F. Golubev and V. Koryanov, “Insectomorphic robot maneuveringon freely rolling balls,”
Journal of Computer and Systems SciencesInternational , vol. 55, no. 1, pp. 125–137, 2016. [25] A. Specian, C. Mucchiani, M. Yim, and J. Seo, “Robotic edge-rollingmanipulation: A grasp planning approach,”
IEEE Robotics and Automa-tion Letters , vol. 3, no. 4, pp. 3137–3144, 2018.[26] Y. Aiyama, M. Inaba, and H. Inoue, “Pivoting: A new method of grasp-less manipulation of object by robot fingers,” in
IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS) , vol. 1, 1993, pp.136–143.[27] Y. Maeda and T. Arai, “Planning of graspless manipulation by amultifingered robot hand,”
Advanced Robotics , vol. 19, no. 5, pp. 501–521, 2005.[28] J. Xiao and X. Ji, “Automatic generation of high-level contact statespace,”
The International Journal of Robotics Research , vol. 20, no. 7,pp. 584–606, 2001.[29] G. Lee, T. Lozano-P´erez, and L. P. Kaelbling, “Hierarchical planning formulti-contact non-prehensile manipulation,” in
IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS) , 2015, pp. 264–271.[30] P. Balatti, F. Fusaro, N. Villa, E. Lamon, and A. Ajoudani, “A collab-orative robotic approach to autonomous pallet jack transportation andpositioning,”
IEEE Access , vol. 8, no. 142, pp. 191–204, 2020.[31] R. Raguram, O. Chum, M. Pollefeys, J. Matas, and J.-M. Frahm,“Usac: a universal framework for random sample consensus,”
IEEETransactions on Pattern Analysis and Machine Intelligence , vol. 35,no. 8, pp. 2022–2038, 2012.[32] Z. Zhang, “Iterative point matching for registration of free-form curvesand surfaces,”
International Journal of Computer Vision , vol. 13, no. 2,pp. 119–152, 1994.[33] J. E. Bobrow, B. Martin, G. Sohl, E. Wang, F. C. Park, and J. Kim, “Op-timal robot motions for physical criteria,”
Journal of Robotic Systems ,vol. 18, no. 12, pp. 785–795, 2001.[34] K. Harada, K. Hauser, T. Bretl, and J.-C. Latombe, “Natural motiongeneration for humanoid robots,” in