A simple script language for choreography of multiple, synchronizing non-anthropomorphic robots
AA simple script language for choreography ofmultiple, synchronizing non-anthropomorphicrobots — Working draft —
Henning Christiansen
Roskilde University , [email protected] October 29, 2018
Abstract
The scripting language described in this document is (in the first place) intended to be usedon robots developed by Anja Mølle Lindelof and Henning Christiansen as part of a researchproject about robots performing on stage.The target robots are expected to appear as familiar domestic objects that take their ownlife, so to speak, and perhaps perform together with human players, creating at illusion ofa communication between them. In the current version, these robots’ common behaviour isdetermined uniquely by a script written in the language described here – the only possibleautonomy for the robots is action to correct dynamically for inaccuracies that arise during aperformance.The present work is preliminary and has not been compared to properly to other researchwork in this area, and the testing is still limited. A first implementation on small Lego Mind-storms based robots is under development by Mads Saustrup Fox as part of his master thesiswork [3]. a r X i v : . [ c s . R O ] D ec ontents Introduction
This document is intended as a reference for CCRS
CRIPT , which is a scripting language for robotsto be used in dance, theatre and other performances. The name of the language is an abbreviationfor something like Choreographing Constrained Robots Script Language. A script in CCRS
CRIPT describes sequences of fixed movement for one or more robots with possible synchronization whenthere are more than one robot. The robots are not assumed to have any abilities for interaction andthey are expected to execute the script faithfully with no independent decision making along theway. The target robots are expected to appear as familiar domestic objects which has no anthro-pomorphic ways of gesturing or communication such as facial expressions, arm waving, gazes etc.They are expected to perform through elementary movements only. Such robots and their potentialhave been characterized and analyzed in a very inspiring scientific article by Bianchini et al [1],which we recommend as compulsory reading for anyone who wants to work in theory or practicewith such robots.The detailed design of CCRS
CRIPT is still under development and its testing is limited, sorefinements and extensions in near future can be expected. A detailed comparison with relatedwork in robotics and dramaturgy is still lacking.
The expected users of the CCRS
CRIPT language are performance directors in a fairly wide sense,which include also students in performance arts. This means that using the language should notrequire any deep background in programming. Scripts in the language must be understandable andusable for the performance director in such a way that he or she can concentrate on the dramaturgicaspects.However, the use of robots means that a technician is likely involved in some way or another,so technical expertise will be ready at hand, which may be helpful, for example, when using pro-gramming language constructs to define new, high-level instructions (as explained in section 3.4).Possible situations of use include that the director, the technician and perhaps performers (dancers,actors, ...) develop the script together.The language design aims at combining simplicity with the necessary expressive power. In-structions take as few parameters as possible, and in cases where a more detailed control may berelevant, this is done through optional parameters. A sequence of movements is described in astraightforward way, and – if needed – a script can be fine-tuned with additional speed and accel-eration annotation. The language is made extensible by embedding it in a standard programminglanguage, so that new and perhaps complex patterns of movements can be added by proceduralabstraction. 3or simplicity, the language is almost freed from explicit timing, and only one syncronizationprimitive is included, expected to be sufficiently general as to handle all interesting cases.
In this version of the language, no interaction or dynamic decisions are anticipated. A script de-scribes fixed sequences of movements for one or more robots, and when once the performanceis going on on the stage, there are no changes. However, as argued elsewhere [2], the over-all dramaturgic effect may appear as a close dialogue and interaction between human and robotperformers. The actual robot implementation should include mechanisms that correct for smallinaccuracies that arise when the robots are moving, due to sliding, wheelspin, imprecision in therobot steering, etc.(It is planned for future versions of CCRS
CRIPT to experiment with different ways of specify-ing limited forms of interactions.)
CCRS
CRIPT is not intended for humanoid or other robots with advanced capabilities in body orfacial expression. They are basically objects that can move around, for example, appearing asfamiliar domestic objects, whose scenic performance and apparent liveness are created by move-ments and perhaps interplay with human performers.As indicated above, the robots are not expected to be very intelligent, technically speaking.They should take no decisions or make any planning, except what is needed for to produce afaithful execution of the given script. Exceptional situations such as collisions are not considered,the robot scripts and entire performance setup are expected to be “debugged” for such before aperformance.A given robot is assumed to have some physical limitations in the movements it can perform.These may be imposed by the actual mounting of wheels (or similar), but may also be addedby design as to produce a certain gestalt. For example, if the robot appears as a car, it may belimited to move as a car moves, even if wheels that are invisible to the audience allow it to movein all directions. These details are not specified in CCRS
CRIPT , but an implementation shouldinclude an algorithm that generates a path from one pose to another, that provides a “naturalway” of moving for the given type(s) of robots. The currently used robots can move forwards andbackwards in two dimensions, and only turn left or right within a certain limited angle. So, forexample, a complex pattern of movements may be needed to have a robot end up in its currentpositions, but turned 180 degrees. In such situations, the different choices of how the result isobtained may have dramaturgically very different effects and should be tailored by the director.This may be done inserting intermediate poses and use a combination of a forward and a backwardmovements. A “pose” in standard robot terms means the position plus angle of its forward direction in some moment of time. .4 Simulator In addition to the implementation on robots, CCRS
CRIPT is also intended to be supported by asimulator. The simulator should produce the same movements as the physical robots, and the twoimplementations must agree to such an extent that reliable time measurements can be made usingthe simulator.It is expected that a script for a given performance is developed by help of the simulator –and in a combination of rehearsals on the stage (perhaps together with humans performers). Thesimulator can be used for “debugging” the script, so no undesired situations, such as collisions, arelikely to occur, and for testing the overall pattern of actions performed by the robots.
The language is independent of a particular implementation, but is intended to be used for bothcontrolling actual robots in physical space and for produce animations in a simulator. These twoimplementations should agree to the extent that in case the robots are not subject to unexpectedincidents, the observable patterns of movements should appear is identical.The robot software should make corrections for small accuracies that arise due to sliding,wheelspin and similar phenomena, so that the robot always ends up in at specified pose (perhapsslightly later that expected). There is no absolute timing that the robots need to keep up to, theprinciple is that “things take the time they will take” (this is a design decision imposed in orderto simplify CCRS
CRIPT ), determined by possible speed and acceleration parameters (e.g., defaultvalues) and the physical conditions. The simulator and robot implementation must agree to suchan extent that reliable time measurements can be made using the simulator.The detailed movements generated by an instruction depend on the robots’ the physical con-straints; this is not specified in CCRS
CRIPT , but an implementation should include an algorithmthat generates a path that provides a “natural way” of moving for a robot with the given gestalt andits physical limitations.The simulator and the robot implementation should both produce the same movements, and thesimulator may give diagnostic messages in case the algorithm used cannot create a “reasonable”path from one place to another – or if the path passes through illegal positions (as defined below).The only way that a director can modify a path from one pose to another is to insert intermediateposes and perhaps switch between forward and backward movements (analogous to parking a carin a narrow space).
CCRS
CRIPT should be embedded a standard programming language in order to provide: • Ways of naming different robots and referring to them; typically by program variables.5
Standard datatypes such as text strings and numbers and abilities to do standard arithmeticand such. • Procedural abstraction that makes it possible to put together sequences of instructions andgive them a new name, thus extending CCRS
CRIPT with new instructions; see examples insection 3.4, below.The current prototype simulator is embedded in Processing that follows Java syntax – which (forthe part used here) is quite mainstream, being more or less identical to C, C++, C color(255,128,128) indicates a pink colour.
The robots move within a scene which is assumed to be rectangular. The x -axis is parallel to theborder of the scene towards the audience, and the y -axis points into the scene as seen from theaudience. The point (cid:104) , (cid:105) is placed middle front in the scene, such that points to the right of themiddle has positive x values and points to the left negative values. The y -values are zero at thefront of the scene and grow positively into the scene; see illustration below.The basic unit is one meter, and other units are available; explained below. The overall dimen-sion of the scene should be defined before any other CCRS CRIPT instructions are use. It may, forexample, be done as follows. float sceneWidth = 10;float sceneDepth = 5;
This defines a scene which is 10 meters wide and 5 meters deep; thus x -coordinates within thescene ranges between − y -coordinates between 0 and 5; any position outside this rangeis called illegal , and it is also possible to define illegal areas within the scene (see section 3.5).There are no mechanisms in the language that prevent a robot moving to an illegal position. A number without explicit units used for a position or distance is interpreted as meters; thefollowing named units are also available: • sw referring to the width of the scene. • hsw referring to half the width of the scene. • sd referring to the depth of the scene. • m for meters (it is redundant, but included for completeness). The director is expected to “debug” the script using a simulator that will issue suitable warnings. Robots in illegalposition can be relevant, e.g., if a robot enters the scene by a door or exits the same way.
CRIPT is embedded in a standard programming language syntax, a multiplicationoperator must be use when using these units; for example, the quarter of the width of the scene maybe indicated as. The coordinate system and coordinates of selected points are shown as follows. ( , ) ( hsw , )( hsw , )( -hsw , sd ) ( hsw , sd )( , sd ) ( hsw/2 , sd/2 )( -hsw/2 , sd/2 ) xx Angles are indicated in degrees, and the following names can be used for absolute directions; thedirection of North points into the scene. • north for the direction into the scene, synonymous with degrees. • east , south , west , northEast , southEast , northWest , southWest analogously. • nne for North-North-East and analogously ene , ese , sse , ssw , wsw , wnw , nnw .Arithmetic can be used, so, e.g., one degree starboard of east can be indicated as east+1 or as . At any given time, a robot has a pose , which is the combination of x and y coordinates and adirection (assumed to be the direction it is heading, or, metaphorically, the direction it is “looking”).So, for example, a robot standing at the very centre of the scene, looking towards the audience hasthe pose (cid:104) , , south (cid:105) . If needed, poses may be treated as data objects, but this is actuallynot necessary for writing scripts. It the programming syntax, a special datatype is used togetherwith a function to construct a pose as a data object: pose(0, 0.5*sd, south) Time is measured in seconds, but the use of explicit timing in CCRS
CRIPT is reduced to a mini-mum: the specified robot movements take the time they will take; time is used explicitly for waitinstructions, e.g., when instructing a robot to stand still for 10 seconds before it goes on with thenext given movement. Notice that such compass angles are different from a traditional mathematical way of considering angles; a math-ematician will typical indicate an angle of zero degrees pointing to what a compasicist will call east (i.e., + ◦ ). Andwhen a mathematician says an angle is in creases, the compasicist will correct him and say its is de creasing. .3 Instructions In this version of CCRS
CRIPT , the physical properties of the robots are not considered explicitly,but they can be specified by a few attributes that may be used by a simulator. The syntax used inthe Processing version of CCRS
CRIPT , is illustrated as follows.
Robot nille = robot("Nille", color(255,128,128));Robot frederik = robot("Frederik", color(128,128,255));
Two robots are defined; they can be referred to later in the script through the program variables nille and frederik ; the parameters to the robot introdiction function robot( · · · ) indicates aname (as a text string) and a colour that may be used by a simulator. (The sample colours are pinkand light-blue.)Each robot need to be given an initial pose, i.e., so to speak be placed into the scene, beforemovement commands give sense. The two robots defined above may be placed as follows. initialPose(nille, hsw/2+1, sd/4, west);initialPose(frederik, hsw/2, sd/4+1, south); This positions are shown in this figure; the heading direction is indicated by the arrow; robot nille is pink, frederik light blue. ( , ) ( hsw , )( hsw , )( -hsw , sd ) ( hsw , sd )( , sd ) The use of poses as data objects is shown in the following code fragment (that does the same asthe above).
Pose p1 = pose(hsw/2+1, sd/4, west);Pose p2 = pose(hsw/2, sd/4+1, south);initialPose(nille, p1);initialPose(frederik, p2);
8t is not enforced that the robots must by placed physically in those positions indicated in the script.It is, of course, recommended to do so; or at least approximatively as a good implementation shouldcorrect for inaccuracies.
Some instructions tell the robot to move to a new, given pose from whatever pose it is at when itstarts executing the instruction. For example: moveTo(nille, -hsw/2, sd/2, south);
Unless otherwise specified, the robot nille is assumed to be at a standstill, it will accelerate toa normal speed, follow the generated path in a forward direction, and finally decelerate and cometo a standstill at the indicated pose. Alternatively, the movement can be done having the robotmoving backwards: moveToBacking(nille, -hsw/2, sd/2, south);
The following, alternative syntax may apply when poses are considers as data objects. moveTo(nille, pose(-hsw/2, sd/2, south));moveToBacking(nille, pose(-hsw/2, sd/2, south));
The remaining instructions specify relative movements. To move forward or backward in thedirection indicated by the current pose is exemplified as follows: move(nille, 2);moveBacking(frederik, 2);
Robot nille moves two meters forward according to its current pose, frederik moves two metersbackwards,There are instruction for moving along a segment of a circle circleRight(nille, 1, 90);
It means: The robot nille should turn to the right by traversing a circle with radius meterand an angle of degrees. There are similar instructions for circling left an circling backwards.Examples: circleRight(nille, 1, 90);circleLeft(nille, 1, 90);circleRightBacking(nille, 1, 90);circleLeftBacking(nille, 1, 90); The angle can be arbitrarily high, so for example the following instruction specifies that nille rotates 100 times without getting anywhere. 9 ircleRight(nille, 1, 36000);
The relative move instructions should executed such that the robot ends up at the pose, where theywould end if all previous move instructions had been executed with mathematical precision, i.e.,the execution of the instructions on the physical robots should adjust for errors in the pose theystart from.
A robot can be told to wait for a given number of seconds, for example: wait(nille, 2);
Each robot executes its own instructions independently of other robots (unless when synchroniza-tion is used; section 3.3.6, below). This means that two sequences of instructions for two differentrobots can be intertwined in different ways without changing the overall behaviour. Consider as anexample the following CCRS
CRIPT fragment. initialPose(nille, hsw/2, 0, north);initialPose(frederik, -hsw/2, 0, north);moveTo(nille, hsw/2, 3, north);moveTo(frederik, -hsw/2, 3, north);moveTo(nille, hsw/2+2, 3, east);moveTo(frederik, -hsw/2+2, 3, east);
The two robots will start moving simultaneously and continue moving in parallel. We may, e.g.,group the instructions for each robot as follows without changing the meaning. initialPose(nille, hsw/2, 0, north);moveTo(nille, hsw/2, 3, north);moveTo(nille, hsw/2+2, 3, east);initialPose(frederik, -hsw/2, 0, north);moveTo(frederik, -hsw/2, 3, north);moveTo(frederik, -hsw/2+2, 3, east);
The robots are assumed to have a standard maximum speed and standard acceleration and decel-eration. Unless otherwise specified, these will be used when interpreting the move instructions.Speed is measured in meters per second and acceleration / deceleration in meters per second .These figures can be set for all robots at the same time or each robot individually. Examples:10 axSpeed(2);acceleration(frederik, 0.5);deceleration(nille, 0.2); Instead of an absolute numbers, the following constants can be used. max : the maximum possible value, determined by the physical robot. std : the standard value for this robot.Dynamic changes in speed during a move instruction may specified by a text string that can beadded as an extra argument to move instruction. The following characters have a special meaninginside these instructions. ! : if appearing as the first character in such a string, it means to start the movement as fastas possible; at the end of a string, it means brake as late and as hard as possible in orderto stop at the specified pose; in any other position, it means break as hard as possible andimmediately start as fast as possible (likely experienced as a sort of jump). = : if appearing as the first character in such a string, it means to continue with the speed fromthe previous instruction; at the end of a string, it means suppress deceleration when thespecified pose is reached. + : increase speed. - : decrease speed.Any other character means continue as you are doing. Control strings consisting of a single “ ! ”,resp. “ = ”, are interpreted as “ !! ”, resp. “ == ”. The use of “ = ” works best when an instruction whosecontrol string ends with “ = ” is followed by another instruction whose control string starts with “ = ”(This is not enforced, and if used differently, the implementation may just ignore the “ = ”.The “ = ” character in control strings can be used to produce a ‘smooth concatenation of differentmove instructions. Example: initialPose(nille, hsw/2, 0, north);moveTo(nille, hsw/2, 3, north, "=");circleRight(nille, 1, 90, "=");moveTo(nille, hsw/2+2, 4, east, "="); The length of a string is not fixed, and the implementation will try to match each character withan equal time interval during the movement. Unless overridden by “ ! ”, a normal acceleration /deceleration and the star/end is assumed. Examples: "++__++____----""!!!!!!!!!!" ++ ); continue a bit withthe obtained speed ( __ ); accelerate more ( ++ ); continue with obtained speed ( ____ ), and thengradually decreasing the speed to normal and the stop by a normal deceleration. The second one islike interpreted as a very little gracious way of jumping along.The following example shows the syntax for using such control strings. moveTo(nille, -hsw/2, sd/2, south, "++__++____---!"); There is only one instruction for synchronization, and it implements the rule “everyone waits forthe slowest one”. synchronize();
This means that each robot finishes all instructions up to the synchronization instruction and stops;when all robots have finished and stopped this way, they all proceed to their next sequence ofinstructions.There are variants of the instructions that allow to specify which robots should synchronize.This may be useful when there are three or more robots in action. Example: synchronize(nille,frederik);
Robots nille and frederik synchronize while all other robots continue whatever they are doing.
The following is an example of how a new instruction be defined using a function definition inProcessing and Java. void steps(Robot rob, int n) {for(int i=1; i<=n; i++) {move(rob, 0.3); moveBacking(rob, 0.3);}}
The new instruction can be used together with other instructions, exemplified as follows. moveTo(nille, 0, 1, south);steps(nille, 7);moveToBacking(nille, 0, sd-1, south, "+++++!"),
Robot nille will go to the centre of the scene, one meter from the edge of the scene, looking atthe audience. Then it steps back and forth 30 cm seven times, appearing to be undetermined, andthen it flees very fast to the back of the scene, and brakes hard one meter from the back cloth.12ere is another definition of a new instruction parameterized by two robots and a position.When called for two robots and a given position, the robots will first find their way to the left, resp.right, of the indicated position; when both have arrived, they perform a ritual in which they mirroreach other. void meetAndGreat(Robot r1, Robot r2, float x, float y) {moveTo(r1, x-1, y, east);moveTo(r2, x+1, y, west);synchronize(r1,r2);moveTo(r1, x-0.25, y, east);moveTo(r2, x+0.25, y, west);wait(r1,1); wait(r2,1);moveToBacking(r1, x-1, y, east);moveToBacking(r2, x+1, y, west);wait(r1,0.5); wait(r2,0.5);}
It may be uses, e.g., as follows in a script. meetAndGreat(nille, frederik, float x, float y);
These instructions affect only execution in a simulator. A forbidden area is a rectangle whichwill be shown with a special colour; the robot is not prevented from entering these areas, but thesimulator will issue a warning. Reference points and grids will be drawn by the simulator in thedepiction of the scene, but serves no other purposes. In the current version of CCRS
CRIPT , thesettings made by these instructions are final: once set, they cannot be un-done. The followingfragment a script shows the current options. grid();referencePoint(color(255,0,0),hsw/2,sd/2);referencePoint(color(0,255,0),-hsw/2,sd/2);forbiddenArea("green stuff", color(192,255,192),-hsw+0.5, sd-1.75, -hsw+3, sd-2 );forbiddenArea("purple stuff", color(192,192,255),0.5,0.5, hsw-2, 1) );
The yet uninhabited scene may be shown as follows, assuming a width of 10 meters and a depthof 5 meters. 13 .6 Possible extension considered for future versions
Relevant if we have robots with different gestalt and-or physical limitations, which may meanalso that the implementations (on the robots and in the simulator) may have different path findingalgorithms. The type of each robot should be indicated when the robot is introduced.
Robot nille = robot("Nille", type1, color(255,128,128));Robot frederik = robot("Frederik", type2, color(128,128,255));
As we have tested different algorithms for find a »best« path from one pose to another, we have ob-served that each have its advantage and may give a different dramaturgic result. An extra argumentmay be added to the moveTo command as shown in the following. For example: moveTo(nille, -hsw/2, sd/2, south, forward);moveTo(nille, -hsw/2, sd/2, south, backward);moveTo(nille, -hsw/2, sd/2, south, smart);
While the two first ones are synonymous with moveTo and moveToBacking commands that areshown above, the smart option may allow the algorithm to combine forward and backward move-ments, which can give shorter and more »natural« paths that reduces the amount of circling.
It may be interesting in the simulator also to include the movements of human performers. Thehumans will in this respect appear as just another type of robot, the only difference is that they are“controlled” only in the simulator. 14 .6.4 Other robot abilities
In case the robots are equipped with blinking lamps, loudspeakers or other sound devices, wavingflags, etc., CCRS
CRIPT should be extended with instructions to use those. This will require a bitof design thinking in case these actions can be performed while the robot is moving (i.e., the samerobot is performing two or more actions at the same time).
As described above, the robots do not react to the actions of a human performer, and the onlyway to create such an illusion is that the human performer knows exactly what the robot will bedoing and acts accordingly. As described elsewhere this can be done in a very convincing way, butrequires top-professional human performers [].It may be interesting to extend the language with facilities that allows an interaction, limited intime and space. For example, the robot may be instruction to stay on a certain line segment andin a given period maintain a constant distance to the object in front of it. If this object is a humandancer, that takes a jump towards the robot, the robot should perform a similar jump backwards.These ideas are currently only at a speculative level, and more experience with robots controlledvia CCRS
CRIPT before such facilities are considered,
A prototype of a simulator has been implemented in Processing. A script is supposed to be includedas the body of a function called manuscript() . The implementation includes an algorithm thatcan calculate a “natural” path between two poses, which fits the movements of a robot which canonly turn left or right within a certain limited angle. Such a path is described as a sequence ofsmall steps, and an additional translation into motor control commands is needed in order to usethis algorithm for the physical robots. An alternative algorithm that always can find a way fromon pose to another, consisting of at most two circle segments and a straight line segment has alsobeen tested.
The following facilities are desirable in a final version of a simulator to be used in performanceproduction. • A straightforward transfer of a script to a robot implementation. • Ability to run simulator and robots concurrently (as to see if the robots actually do what theyare supposed to). 15
Starting and stopping the simulation, normal and reverse simulation, slow and fast simula-tion. • Automatic annotation of the given script with timing information, • Changing viewpoint, between seeing things from above and perspective views, as soon fromthe audienceIt has not been considered whether it is interesting to use high quality, 3D graphics (e.g., usingUnity 3D), and if arbitrary viewpoints are useful. Perhaps a schematic view will still give the bestoverview.
References [1] Samuel Bianchini, Florent Levillain, Armando Menicacci, Emanuele Quinz, and ElisabettaZibetti. Towards behavioral objects: A twofold approach for a system of notation to designand implement behaviors in non-anthropomorphic robotic artifacts. In Jean-Paul Laumond andNaoko Abe, editors,
Dance Notations and Robot Motion, 1st Workshop of the Anthropomor-phic Motion Factory, at LAAS-CNRS, Toulouse, France, 13-14 November, 2014 , volume 111of
Springer Tracts in Advanced Robotics , pages 1–24. Springer, 2014.[2] Henning Christiansen and Anja Mølle Lindelof. Robots on stage.