Software Engineering for Robotic Systems:a systematic mapping study
SS OFTWARE E NGINEERING FOR R OBOTIC S YSTEMS : ASYSTEMATIC MAPPING STUDY
A P
REPRINT
Marcela G. dos Santos
SmArtSE Research TeamUniversité du Québec à Chicoutimi555, boulevard de l’Université, G7H 2B1Chicoutimi, QC, Canada [email protected]
Fabio Petrillo
SmArtSE Research TeamUniversité du Québec à Chicoutimi555, boulevard de l’Université, G7H 2B1Chicoutimi, QC, Canada [email protected]
February 26, 2021 A BSTRACT
Robots are being applied in a vast range of fields, leading researchers and practitioners to write tasksmore complex than in the past. The robot software complexity increases the difficulty of engineeringthe robot’s software components with quality requirements. Researchers and practitioners haveapplied software engineering (SE) approaches and robotic domains to address this issue in the lasttwo decades. This study aims to identify, classify and evaluate the current state-of-the-art SoftwareEngineering for Robotic Systems (SERS). We systematically selected and analyzed 50 primarystudies extracted from an automated search on Scopus digital library and manual search on thetwo editions of the RoSE workshop. We present three main contributions. Firstly, we provide ananalysis from three following perspectives: demographics of publication, SE areas applied in roboticsdomains, and RSE findings. Secondly, we show a catalogue of research studies that apply softwareengineering techniques in the robotic domain, classified with the SWEBOK guide. We have identified5 of 15 software engineering areas from the SWEBOK guide applied explicitly in robotic domains.The majority of the studies focused on the development phase (design, models and methods andconstruction). Testing and quality software areas have little coverage in SERS. Finally, we identifyresearch opportunities and gaps in software engineering for robotic systems for future studies. K eywords Robotics · Software Engineering · Systematic literature review · Systematic mapping
Robotic systems (RS) are a complex combination of hardware and software components that interact with the realworld. Besides, RS components must be correctly integrated to enable the system to perform as expected. Nowadays,different domains use robots; from industries to healthcare, passed by a critical mission, robots become more present inour world [1].Recent systematic mappings address software engineering areas in the RS in different SE areas as architecture [1],requirements engineering [2], safety for mobile robotic systems. Besides, there are surveys as [3], a survey focusedon a specific middleware for robotic systems, and [4] that performs a study focusing on software engineering in aservice robotics perspective. However, those previous studies neither address SE in general for robotic systems norSWEBOK [5] areas.The aim for this study is to identify, classify and evaluate the software engineering systematic areas used in the roboticdomain based on the SWEBOK guide. To achieve this goal, we systematically applied the mapping study methodology proposed by Kitchenham et al. [6]. a r X i v : . [ c s . R O ] F e b PREPRINT - F
EBRUARY
26, 2021The main contributions of this study are: (i) a systematic mapping of software engineering in robotic systems; (ii) acatalogue offering research studies about the utilization of software engineering in the robotic domain; and (iii) researchopportunities and gaps in software engineering to robotic systems for future studies. The audience of this study iscomposed of both robotics researchers and roboticists interested in contributing to this research area and applyingsoftware engineering techniques to improve robotic systems.Our study is organized as follows. Section 2 defines the concepts of robotic systems and SWEBOK guide. Section 3describes in detail the adopted study design. Section 4 presents the results from our analysis. Section 5 discusses themain findings and the threats to validity. Section 6 shows related studies. Section 7 synthesizes the final remarks andfuture work.
From a software perspective, robotic systems have two layers: control and application layers. In the control layer, thereare the drivers responsible for interacting directly with the hardware. The application layer is the software componentsresponsible for defining robots’ desired behaviour [1].Typical characteristics of the robotic software are embedded, concurrent, real-time, distributed, data-intensive. Besides,robotics software needs to meet some quality requirements, such as quality aspects requirements, such as safety,reliability, and fault-tolerance [7]. These characteristics and quality requirements are also present for software systemsin other domains, such as avionics, automotive, factory automation, telecommunication and even large-scale informationsystems.Software engineering is the systematic application of scientific and technological knowledge, methods, and experienceto the design, implementation, testing, and documentation of software [8]. In 2004, the first version of Guide tothe Software Engineering Body of Knowledge (SWEBOK) was a mechanism for acquiring knowledge in softwareengineering professionals [5].The SWEBOK is in version 3.0, and there are 15 Knowledge Areas (KA): Software Requirements, Software Design,Software Construction, Software Testing, Software Maintenance, Software Configuration Management, SoftwareEngineering Management, Software Engineering Process, Software Engineering Models and Methods, SoftwareQuality, Software Engineering Professional Practice, Software Engineering Economics, Computing Foundations,Mathematical Foundations, Engineering Foundations [5]. Although conventional and robotic systems are different,most studies about software engineering and robotics apply some SE techniques described in SWEBOK. For this reason,we decided to use it as the base for our classification framework.
The study followed the guidelines for systematic mapping studies [6]. Figure 1 summarizes the steps of our study.
We refined our research goal into three research questions:
RQ1: What are the demographics of publication of research studies about SE areas applied in the roboticsdomain?
By answering this research question, we aim to understand how the research area is studied and relevantvenues where researchers published their results on the topic.
RQ2: What are the SWEBOK areas mentioned in the robotics domain?
We studied the SWEBOK areas appliedin robotics domains, including the 15 SWEBOK areas defined in the guideline [5].
RQ3: What are the main findings of SERS?
We aim to provide information to help researchers with evidence-basedindications about the directions related to software engineering practices.
Following the systematic review process in software engineering guidelines, the first step is creating a search query [9].Our search query is: ((robot*) AND ("software engineering")) PREPRINT - F
EBRUARY
26, 2021
Scopus1657studies
IC3 312studies IC1 307studies IC4 64studies34stuidesRoSE11studies 52studiesIC3Snowballingforward andbackward IC5 23 studies50studiesRoSE16studies IC3 Automated searchManual search
Figure 1: Overview of the search strategy and study selection processesWe defined the inclusion and exclusion criteria during the protocol definition to obtain a set of relevant studies to answerour RQs. We include a study if it satisfies all inclusion criteria and will discard it if the study met any exclusion criteria.
Inclusion criteria (IC) • IC1:
The study must be a primary study;•
IC2:
The paper must be an article or conference paper;•
IC3:
The study mentions software engineering topic in robotic domain;•
IC4:
The study was published in a venues that is ranked as A*, A and B for CORE Conference Ranking [10];•
IC5:
The study must be a full paper.
Exclusion criteria (EC) • EC1:
The study published as an abstract;•
EC2:
The study is not written in English;•
EC3:
The study is a short paper, keynote, tutorial, challenge, and showcase;•
EC4:
The study does not mention software engineering topic in robotic domainIn our study, we performed a hybrid search strategy proposed by Mourao et al. [11] that combine the search in adigital library and a parallel snowballing forward and backward. A forward snowballing refers to identifying newpapers based on papers that cite citing the paper examined, and backward snowballing is the process to evaluate thereferences for the paper in the set [12]. Figure 1 shows an overview of the search strategy and study selection thatwe performed in our study. A total of 1657 studies were turned from the automated search execution. We removedconference announcements and applied the selection criteria (IC and EC) on titles and abstracts, leading to 23 candidatestudies. Next, we combined the results of a manual search performed in the last editions of the International Workshopon Robotics Software Engineering (RoSE) after applying IC3, remaining in a single list with 34 studies. Finally,we performed forward and backward snowballing in the set of paper and with the results and applied exclusion andinclusion criteria on title and abstract leading to 16 studies that were added to the seed set of 34 studies. Remaining ainitial final set of studies with 50 papers. The studies included in the review are listed in Table 1.3
PREPRINT - F
EBRUARY
26, 2021
To extracted data from each primary study, we create a data extraction form. This form was created as a spreadsheetwith specific fields to support our data extraction activity.First, we collected keywords and concepts through the full reading of the papers and applied the classification in aprimary set of 10 papers. When the researchers found some divergence, they discussed, evaluated and refined theframework. The process ended when there were no primary studies to analyze left.The classification framework is composed of three different categories, one for each research question.
Publications Demographics (RQ1).
The parameters that we use collect data about publications trends are: publicationtype and publication venue.
Software Engineering areas in Robotics Domain (RQ2).
To answer research question 1 (RQ2), we used the softwareengineering body of knowledge (SWEBOK). We are using
SWEBOK Guideline version 3 that is composed of 15knowledge areas.
Main findings of RSE (RQ3).
We highlight the results and conclusions of all studies evaluated. After that, we createdclusters to conclude that they have similarities and described important aspects of robotics software engineering. Wealso eliminated the specific findings for one evaluated study because, in this case, the finding is not a global findingrelated to robotics software engineering.
We summarize the extracted data with a combination of content analysis and narrative synthesis. We did not identifyinconsistencies within the data. We present the data synthesis and our findings in Section 4 with a catalogue offeringresearch studies about the utilization of software engineering in the robotic domain and in what SWEBOK KnowledgeArea the study is concentrated (Table 1).
We performed a hybrid search strategy that combines automated, manual and snowballing. The manual search wasperformed on the Robotics Software Engineering Workshop (RoSE) realized editions (2018 and 2019). For this reason,we have 11 studies from a workshop even though a workshop does not have the Core Rank of at least B.RoSE workshop studies are in our data set because it is a workshop focused on Robotics Software Engineering.Although the publication years are not a category that we use to answer the RQ1, the distribution in years is valuable tounderstand the importance of venues as RoSE. In 2018, which is the year of the first RoSE edition, we have the addedof 8 studies, and in 2019 we have in our set of studies only studies were published only in the
RoSE . Publications type
We classified the primary studies by type (conference or journal paper). The publication typedistribution is 26 (11 from RoSE) studies published as a conference paper and 24 in journals. If we evaluated the set ofstudies without the papers from RoSE, we have 15 studies from conference papers and 24 from journals.
Publications venues
The studies evaluated are distributed in 19 different venues (Table 2) which research areas areComputer Science and Robotics. We observed a fragmentation in terms of publication venues, 10 of the venues haveRobotics as the main research field, and nine are focused on Computer Science, and no venue addresses the intersectionbetween robotics and software engineering. We observed that the significant number of studies published in one venuewas 4, and it happened in journals whose main research field is robotics. Besides, three conferences ranked as A* in theset of venues, and we evaluated six studies published in these conferences.
Observation 1 - Demographics of publication
There is a distribution in venues with two different research areas,
Computer Science and Robotics . Weobserved that RoSE play a critical role to the development for a interdisciplinary research field as RoboticsSoftware Engineering, besides the studies are published in journals more than in conferences.4
PREPRINT - F
EBRUARY
26, 2021Table 1: List of primary studies
ID Ref. Title SWEBOK Area
P2 [13] Agile Robot Development (aRD): A pragmatic approach to robotic software. SCP7 [14] Robotics library: An object-oriented approach to robot applications. SCP9 [15] Runtime models for automatic reorganization of multi-robot systems. SCP14 [16] A component-oriented programming for embedded mobile robot software. SCP15 [17] A new paradigm for open robotics research and education with the C++ OOML. SCP24 [18] Visualizing the Hidden “Variables in Robot Programs”. SCP25 [19] Towards Systematic Engineering of Collaborative Heterogeneous Robotic Systems. SCP26 [20] Novice-Friendly Multi-Armed Robotics Programming. SCP29 [21] On the Software Engineering Challenges of Applying Reactive Synthesis toRobotics. SCP32 [22] Engineering Safety in Swarm Robotics. SCP38 [23] RoboNetSim: An integrated framework for multi-robot and network simulation. SCP40 [24] Development of a flexible language for mission description for multi-robot missions. SCP45 [25] Evaluating CoBlox: A comparative study of robotics programming environments foradult novices. SCP46 [26] Modeling of Movement Control Architectures based on Motion Primitives usingDomain-Specific Languages SCP49 [27] RoboFlow: A flow-based visual programming language for mobile manipulationtasks SCP18 [28] Controlling Kuka industrial robots. SC, SDP33 [29] Towards Code-Aware Robotic Simulation: Vision Paper. SC, SDP35 [30] An architectural framework for modeling teleoperated service robots. SC, SDP37 [31] Internet control architecture for internet-based personal robot. SC, SDP41 [32] EmSBot A modular framework supporting the development of swarm roboticsapplications. SC, SDP42 [33] Introducing safety requirements traceability support in model-driven development ofrobotic applications. SC, SDP44 [34] Design patterns for exploring and prototyping human-robot interactions. SC, SDP20 [35] Software product line engineering for robotic perception systems. SC, SMMP22 [36] A framework-based approach for fault-tolerant service robots. SC, SD, STP1 [37] SafeRobots: A model-driven Framework for developing Robotic Systems. SC, SD, SMMP31 [38] Designing Control Software for Robot Swarms: Software Engineering for theDevelopment of Automatic Design Methods. SC, SD, SMMP34 [39] Towards Rapid Composition with Confidence in Robotics Software. SC, SQ, STP21 [40] Building the executive system of autonomous aerial robots using the Aerostackopen-source framework. SC, SQ, SDP3 [41] Service oriented robotic architecture for space robotics: Design, testing, and lessonslearned. SDP6 [42] Towards an architecture for semiautonomous robot telecontrol systems. SDP8 [43] UML-based service robot software development: A case study. SDP11 [44] Component specifications for robotics integration. SDP17 [45] Service robot for the elderly: Software development with the COMET/UMLmethod. SDP27 [46] Checking Consistency of Robot Software Architectures in ROS. SDP43 [47] Service robot system with an informationally structured environment. SDP47 [48] Re-engineering software architecture of home service robots: A case study SDP36 [49] Experimental Testbed for Large Multirobot Teams: Verification and Validation. SDP4 [50] Designing a meta-model for a generic robotic agent system using Gaia methodology. SD, SMMP5 [51] EUD-MARS: End-user development of model-driven adaptive robotics softwaresystems. SD, SMMP10 [52] Model-driven behavior specification for robotic teams. SD, SMMP13 [53] Software toolkit for modeling, simulation, and control of soft robots. SD, SMMP16 [54] Design of service robots: Experiences using software engineering. SD, SMMP19 [55] Model-driven analysis and design for software development of autonomousunderwater vehicles. SD, SMMP23 [56] Object-oriented concepts for modular robotics systems. SD, SMMP28 [57] A Use Case in Model-Based Robot Development Using AADL and ROS. SD, SMMP30 [58] Using Physical Quantities in Robot Software Models. SD, SMMP39 [59] An interaction-driven approach to identifying functional behaviors of service robotsystems. SD, STP12 [60] Model-based testing of a real-time adaptive motion planning system. STP50 [61] Automated testing support for a robot tape library STP48 [62] Reliability analysis of mobile robots SQ, STSC= Software Construction, SD= Software Design, SMM= Software Models and Methods, SQ= Software Quality,ST= Software Testing PREPRINT - F
EBRUARY
26, 2021Table 2: List of Venues
Venue Title Type CORERanking Number of studies
Information Sciences Journal A 3Journal of Field Robotics Journal A 1Science of Computer Programming Journal A 1International Journal of Advanced Robotic Systems Journal B 4IEEE Robotics and Automation Magazine Journal B 4Autonomous Robots Journal B 3Advanced Robotics Journal B 2Robotica Journal B 2International Journal of Semantic Computing Journal B 1Robotics and Autonomous Systems Journal B 2International Conference on Software Engineering Conference A* 3Conference on Human Factors in Computing Systems Conference A* 2International Conference on Autonomous Agents andMulti-agent Systems Conference A* 1IEEE International Conference on Intelligent Robots andSystems Conference A 3International Conference on Engineering of ComplexComputer Systems Conference A 1International Symposium on Software ReliabilityEngineering Conference A 1IEEE International Conference on Robotics and Automation Conference B 3Technology of Object-Oriented Languages and Systems Conference B 1
We aim with RQ2 to provide an overview for researchers and practitioners about the established software engineeringareas studied in robotic domains. As shown in Table 3, 5 of 15 SWEBOK knowledge areas appear on the studiesevaluated for us. Table 3: List of included studies for each SWEBOK knowledge areas
Knowledge Area References Number of studiesSoftware Design
P1, P3, P4, P5, P6, P8, P10, P11, P12, P13, P16, P17,P18, P19, P21, P22, P23, P27, P28, P30, P31, P33, P35,P36, P37, P39, P41, P42, P43, P44, P47 31
Software Construction
P1, P2, P6, P7, P9, P14, P15, P18, P20, P21, P22, P24,P25, P26, P29, P31, P32, P33, P34, P35, P38, P40, P41,P42, P44, P45, P46, P49 27
Software Models andMethods
P1, P4, P5, P10, P13, P16, P19, P20, P23, P28, P30, P31 12
Software Testing
P12, P22, P34, P39, P48, P50 6
Software Quality
P21, P34, P48 3
The SWEBOK areas was the framework used to classify the studies. Thus, the SWEBOK is more cited in our set ofstudies was
Software Design with 60% of the studies, and the techniques applied in the software design area are UML,model-driven development, component-based software engineering, architecture and description language.The second SWEBOK area in our rank was
Software Construction (54%), with most of the studies proposing aframework for solving software problems in the robotics domain.
Software Models and Methods with 24% of the studies is the third SWEBOK area. The SE techniques applied in thisarea are the meta-model, analysis of models, framework model.6
PREPRINT - F
EBRUARY
26, 2021
Software Testing (12%) and
Software Quality (6%) are the two of five SWEBOK areas cited with fewer studiesevaluated. The SE techniques investigated are test generation method, code quality standard, requirement analysis and afault-tolerance architecture.
Observation 2 - SWEBOK areas in Robotics domain
The majority of the studies are concentrated in 3 areas that performed activities inherent to robotics softwaredevelopment ( software design , construction and models and methods ) . Only 18% of the studies performedactivities related to testing and quality, and the evaluated studies do not cover 10 SWEBOK areas, as shown inFigure 2. Software DesignSoftware ConstructionSoftware Models and MethodsSoftware TestingSoftware Quality0% 20% 40% 60%
Figure 2: Number of studies and SE technique applied in Robotics Domain.
With the answers for RQ3, we aim to provide information to help researchers with evidence-based indications about thedirections related to software engineering practices that are used.The first finding is related to the evidence in our set of studies that software engineering practices improve roboticsystems quality.
According to Aceres et al. [54], software engineering contributes to the quality improvement ofrobotic systems and reduces the effort related to the development. Besides, in the study performed by Kim et al. [43], itis highlighted that it is impossible to resolve integration and developing issues in robotic systems without systematic andcomprehensive software development method. Finally, the standard development is described as a key to the usabilityand sustainability of the robotic assets [41].Another important finding is how software engineering practices can help disseminate development for roboticsystems . There is evidence that applying approaches based on model-driven development can empower not professionalprogrammers to develop robotics software without requiring advanced technical skills [51]. In the study performed byGerasimou et al. [19], the Domain Specific Language concepts allow both software engineers and robotic experts towrite robots software with more facility. Gonzales performed a study that allows engineers to create robotic platformsor other physical designs to exchange and collaborate with other researchers. [17].
Observation 3 - Main Findings about SER
There is a consensus by the research community that software engineering techniques and approaches canimprove the quality of robot software. 7
PREPRINT - F
EBRUARY
26, 2021
In this study, we evaluated Robotics Software Engineering to elicit how the SE areas are applied in the robotic domainsand the gaps/opportunities for future work in the intersection of these two research fields.We highlight two aspects of the study’s demographics. First, we observed a distribution related to the publicationsvenues (Table 2), the evaluated studies are published in 19 different venues. The studies are published in venueswhere the main research field is Robotics (9) or Computer Science(9), further the workshop RoSE. Second, mostjournals focus on robotics, and most conferences focus on computer science. The venues ranked in the highestlevel are three conferences: International Conference on Software Engineering Conference, Conference on HumanFactors in Computing Systems Conference, International Conference on Autonomous Agents and Multi-agent SystemsConference.These facts lead us to conclude that there is not a concentration in a unique venue or area. Moreover, through observation,a dissonance between the methodology applied in the evaluated studies published in robotic venues concerned softwareengineering techniques and approaches. For example, in the studies that use model-driven development, 80% do nothave empirical study as is trivial in a software engineering study.In the context of SE Areas, we observed that the evaluated studies cited 5/15 SWEBOK Knowledge Areas:
SoftwareDesign, Software Construction, Software Models and Methods, Software Testing and Software Quality . The top 3 areasmentioned are related to the Design and Implementation phase of the traditional approach of system development:
Software Design, Software Construction and Software Models & Methods . However, 10/15 SE areas are not mentioned,and 5 of these studies are deeply researched by the software engineering community:
Software Requirements, SoftwareMaintenance, Software Configuration Management, Software Engineering Management, Software Engineering Process .We believe that there are gaps and research opportunities in these 5 SE areas for robotic domains.The studies that cited topics related to the
Software Models and Methods area also applied some topics related to theSoftware Design area (11/29). Besides, only five studies (P5, P10, P19, P20, P28) in
Software Models and Methods declare they are working with Model-driven development (MDD). However, 80% of the studies related to MDD haveevaluation issues. For instance, in the study performed by [54], it is unclear how the authors evaluated the part of theproposal related to MDD. This observation suggests a lack of empirical evaluation in the projects that apply MDD . Software construction is the second area more cited in the evaluated studies. According to the SWEBOK [5], softwareconstruction is a SE area related to coding, verification, unit testing, integration testing, and debugging. However, inthe studies evaluated, the technique more applied is software construction tools with a focus on coding as the studiesperformed by Rickert et al. [14] that describe a library approach named Robotics Library (RL), and by Alvarez etal. [30] which purpose is presenting a reference architectural framework for service robot control applications.
Software testing was cited only in 12% of the studies. This result corroborated with other studies, for example, in thesurvey about testing robotic system performed Afzal [63] et al. . In this study, the author mentioned that robots interactwith a physical component, which is a challenge for testing robotic systems.Besides, only 6% of the studies evaluated are concerned
Software Quality . We highlight that for our classification, itwas important not only to cite quality requirements but also to develop topics related to
Software Quality topics such assoftware quality fundamentals, software quality management processes, practical considerations, and software qualitytools [64].Threats to validity usually happen in a mapping study, and it was not different in our study. We highlight some of thosethreats and the mechanism that we applied to address them.First, the main limitation is that our search string is restrictive concerning software engineering terms. This choicecould lead us to primary studies that are not representative of state-of-the-art robotics software engineering. We mitigatethis threat by applying a hybrid search strategy proposed by Mourao et al. , and Wholin et al. [11, 12], with automatedand manual search in the first stage and backward-forward snowballing.Second, we considered only peer-reviewed studies published in venues with CORE Conference Ranking at least B [10];this potential bias did not impact our study since the studies evaluated were reviewed in a rigorous process, which is arequirement for high-quality.Another limitation of this work is the established categories. We could identify other categories if we decided, forexample, to classify the approaches and techniques. However, we opted to follow the SWEBOK guideline because it isa known guideline used in SE. Besides, we could answer other RQs and provide a broad overview of SE for roboticsystems. 8
PREPRINT - F
EBRUARY
26, 2021
Ahmad et al. [1] performed a systematic mapping study on software architectures for robotic systems on a set of 56primary studies. The authors identified eight themes that support architectural solutions, the progress from object-oriented to component-based, and cloud robotics as an emerging solution. Our study differs from [1] in the followingterms: (i) we apply a more general search process by considering studies related to all SE areas, so we use a search stringwith the expression "software engineering" instead of using specific terms, for example, architecture, frameworkor development; (ii) we use a known guideline,
SWEBOK guide [5], as the framework to classify the studies.In [65], Bozhinoski et al. performed a systematic mapping study on safety for mobile robotic systems with a softwareengineering perspective. They analyzed 58 primary studies. Their study focuses on existing solutions that use softwareengineering methods or techniques to manage mobile robotic systems’ safety. Our study does not analyse solutions in aspecific type of robotic system. For this reason, we use the term robotics* in our search strategy/ Alos, our studyaims to analyze and identify all the studies that apply
Software Quality techniques, as software quality fundamentals,software quality management processes, practical consideration, software quality tools [64].Malavolta et al. [3] performed a study to elicit evidence-based architectural guidelines for open-source ROS-basedsoftware for robots. They constructed a data-set of 335 GitHub repositories containing real ROS-based systems and anonline survey. Their study plays an important role in the mapping of Software Engineering for Robotics, given theincrease of the ROS utilization by roboticists and architecture robotic researchers. Our study differs from their studybecause (i) we apply an empirical method, i.e., systematic mapping; (ii) we do not focus on a specific middleware(ROS).Garcia [4] have et al. reported a survey study of robotics software engineering from a service robotics domainsperspective. The survey evaluated the robotics system’s state through 18 semi-structured interviews with industrialpractitioners and a survey with 156 respondents. The authors provide a comprehensive picture of software engineeringpractices in the robotics domain, specific characteristics of robotics software engineering and the usual challengeswith adopted solutions. The differences between the work by Garcia et al. and our study is: (i) we apply a differentmethodology in our study, i.e., we apply a systematic mapping using the guideline proposed by [9]; (ii) we do notfocus on a type of robots; (iii) our study has data from academic publications. We considerate our study and thestudies performed by Malavolta [3] et al. , and by Garcia [4] et al. as complementary studies about robotics softwareengineering.
We evaluated 50 primary studies and produced a systematic mapping on SE areas in the robotics domain based onthe SWEBOK guide, and we identify gaps to lead research in the robotics software development. Nineteen differentvenues(journals and conferences) concentrated on the studies evaluated. The majority of the journals focus on Robotics,and the venues with a higher level in the Core Ranking are Computer Science conferences. We evaluated 24 journalstudies, 15 conference studies and 11 studies from the RoSE workshop.We observe that the academy’s effort to apply SE techniques with 5/15 SWEBOK areas cited in the studies in the lastyears. However, most of the studies (Software design 60%, Software Construction 54% and Software Models andMethods 24%) are related to the development phase. Other areas, for example, software testing (12%) and quality(4%),are not broadly applied in robotic domains or published.Besides, there are ten areas in the SWEBOK that are not mentioned in the evaluated studies. We speculate that thereare 5/10 SE area that is not cited but can significantly impact robotic systems’ software development. We believe thatthere are SE areas not properly established yet in robotic domains, a research opportunity that needs to be addressed byresearch robotic and software engineering.Robotic software development focuses on integrating and reusable components [4], so model-driven and component-based using models, methods, and frameworks to construct software solutions are broadly applied. However, throughour study, we observed a gap in empirical studies that evaluated the utilization of model-driven robotic domains.We hope with our study to contribute to the movement of synergy between robotics and software engineering communi-ties. Based on our discussion, we believe that there is a gap in software testing and quality for robotics systems, andthe community must focus on these SE areas to improve robotics software components. As future work, we intend toperform our research query in three other software engineering digital libraries: ACM Digital Library, IEEE Xplore,and Web of Science. Besides that, we aim to perform a cross-analysis among our study and surveys presented in therelated works [3, 4]. 9
PREPRINT - F
EBRUARY
26, 2021
References [1] Aakash Ahmad and Muhammad Ali Babar. Software architectures for robotic systems: A systematic mappingstudy.
Journal of Systems and Software , 122:16–39, 2016.[2] Danyllo Albuquerque, J. Castro, Sarah Ribeiro, and T. Heineck. Requirements Engineering for Robotic System:A Systematic Mapping Study.
WER , 2017.[3] Ivano Malavolta, Grace A Lewis, Bradley Schmerl, Patricia Lago, and David Garlan. How do you Architect yourRobots? State of the Practice and Guidelines for ROS-based System. In
Proceedings of the 42nd InternationalConference on Software Engineering: Software Engineering in Practice , 2020.[4] Sergio García, Daniel Strüber, Davide Brugali, Thorsten Berger, and Patrizio Pelliccione.
Robotics SoftwareEngineering: A Perspective from the Service Robotics Domain , page 593–604. Association for ComputingMachinery, New York, NY, USA, 2020.[5] Pierre Bourque, Richard E. Fairley, and IEEE Computer Society.
Guide to the Software Engineering Body ofKnowledge (SWEBOK(R)): Version 3.0 . IEEE Computer Society Press, Washington, DC, USA, 3rd edition, 2014.[6] Barbara Kitchenham and Pearl Brereton. A systematic review of systematic review process research in softwareengineering.
Inf. Softw. Technol. , 55(12):2049–2075, December 2013.[7] C. Schlegel, T. Hassler, A. Lotz, and A. Steck. Robotic software systems: From code-driven to model-drivendesigns. In , pages 1–8, 2009.[8] Systems and software engineering — Vocabulary, 2017.[9] Kitchenham BA and Stuart Charters. Guidelines for performing systematic literature reviews in softwareengineering. 2, 01 2007.[10] Computing Research & Education. Core rankings portal.[11] Erica Mourão, João Felipe Pimentel, Leonardo Murta, Marcos Kalinowski, Emilia Mendes, and Claes Wohlin. Onthe performance of hybrid search strategies for systematic literature reviews in software engineering.
Informationand Software Technology , 123:106294, 2020.[12] Claes Wohlin. Guidelines for snowballing in systematic literature studies and a replication in software engineering.In
Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering ,EASE ’14, New York, NY, USA, 2014. Association for Computing Machinery.[13] Berthold Bäuml and Gerd Hirzinger. Agile Robot Development (aRD): A pragmatic approach to robotic software.
IEEE International Conference on Intelligent Robots and Systems , pages 3741–3748, 2006.[14] Markus Rickert and Andre Gaschler. Robotics Library : An Object-Oriented Approach to Robot Applications.2017.[15] Christopher Zhong, Nichols Hall, Scott A Deloach, and Nichols Hall. Runtime Models for Automatic Reorganiza-tion of Multi-Robot Systems. pages 20–29, 2011.[16] Dayang N.A. Jawawi, Rosbi Mamat, and Safaai Deris. A component-oriented programming for embedded mobilerobot software.
International Journal of Advanced Robotic Systems , 4(3):371–380, 2007.[17] Alberto Valero-gómez Juan González-gómez. A new paradigm for open robotics research and education with theC ++ OOML. pages 233–249, 2013.[18] David C. Shepherd, Nicholas A. Kraft, and Patrick Francis. Visualizing the ’hidden’ variables in robot programs.
Proceedings - 2019 IEEE/ACM 2nd International Workshop on Robotics Software Engineering, RoSE 2019 , pages13–16, 2019.[19] Simos Gerasimou, Nicholas Matragkas, and Radu Calinescu. Towards systematic engineering of collaborativeheterogeneous robotic systems.
Proceedings - 2019 IEEE/ACM 2nd International Workshop on Robotics SoftwareEngineering, RoSE 2019 , pages 25–28, 2019.[20] Nico Ritschel, Reid Holmes, Ronald Garcia, and David Shepherd. Novice-friendly multi-armed robotics program-ming.
Proceedings - 2019 IEEE/ACM 2nd International Workshop on Robotics Software Engineering, RoSE 2019 ,pages 29–32, 2019.[21] Shahar Maoz and Jan Oliver Ringert. On the software engineering challenges of applying reactive synthesis torobotics.
Proceedings - International Conference on Software Engineering , pages 17–22, 2018.[22] Giovanni Beltrame, Ettore Merlo, Jacopo Panerati, and Carlo Pinciroli. Engineering safety in swarm robotics.
Proceedings - International Conference on Software Engineering , pages 36–39, 2018.10
PREPRINT - F
EBRUARY
26, 2021[23] Michal Kudelski, Luca M. Gambardella, and Gianni A. Di Caro. RoboNetSim: An integrated framework formulti-robot and network simulation.
Robotics and Autonomous Systems , 61(5):483–496, 2013.[24] Daniel Castro Silva, Pedro Henriques Abreu, Luís Paulo Reis, and Eugénio Oliveira. Development of a flexiblelanguage for mission description for multi-robot missions.
Information Sciences , 288(1):27–44, 2014.[25] David Weintrop, Afsoon Afzal, Jean Salac, Patrick Francis, Boyang Li, David Shepherd, and Diana Franklin.Evaluating CoBlox. (February 2019):1–1, 2018.[26] Arne Nordmann, Sebastian Wrede, and Jochen Steil. Modeling of movement control architectures based onmotion primitives using domain-specific languages.
Proceedings - IEEE International Conference on Roboticsand Automation , 2015-June(June):5032–5039, 2015.[27] Sonya Alexandrova, Zachary Tatlock, and Maya Cakmak. RoboFlow: A flow-based visual programming languagefor mobile manipulation tasks.
Proceedings - IEEE International Conference on Robotics and Automation ,2015-June(June):5537–5544, 2015.[28] Houxiang Zhang and Massimiliano Fago. Controlling Kuka Industrial Robots: Flexible Communication InterfaceJOpenShowVar. (December), 2015.[29] John-paul Ore, Carrick Detweiler, and Sebastian Elbaum. Towards Code-Aware Robotic Simulation Vision Paper.pages 40–43, 2018.[30] An architectural framework for modeling teleoperated service robots.
Robotica , 24(4):411–418, 2006.[31] Kuk Hyun Han, Sinn Kim, Yong Jae Kim, and Jong Hwan Kim. Internet control architecture for internet-basedpersonal robot.
Autonomous Robots , 10(2):135–147, 2001.[32] Long Peng, Fei Guan, Luc Perneel, and Martin Timmerman. EmSBot A modular framework supporting thedevelopment of swarm robotics applications.
International Journal of Advanced Robotic Systems , 13(6):1–15,2016.[33] Pedro Sánchez, Diego Alonso, Francisca Rosique, Bárbara Álvarez, and Juan A. Pastor. Introducing safetyrequirements traceability support in model-driven development of robotic applications.
IEEE Transactions onComputers , 60(8):1059–1071, 2011.[34] Allison Sauppé and Bilge Mutlu. Design patterns for exploring and prototyping human-robot interactions.
Conference on Human Factors in Computing Systems - Proceedings , pages 1439–1448, 2014.[35] Davide Brugali and Nico Hochgeschwender. Software product line engineering for robotic perception systems.
International Journal of Semantic Computing , 12:89–107, 03 2018.[36] Heejune Ahn, Woong Kee Loh, and Woon Young Yeo. A framework-based approach for fault-tolerant servicerobots.
International Journal of Advanced Robotic Systems , 9:1–10, 2012.[37] Arunkumar Ramaswamy, Bruno Monsuez, and Adriana Tapus. SafeRobots: A model-driven Framework fordeveloping Robotic Systems.
IEEE International Conference on Intelligent Robots and Systems , (Iros):1517–1524,2014.[38] Darko Bozhinoski, Université Libre, Mauro Birattari, and Université Libre. Designing Control Software for RobotSwarms: Software Engineering for the Development of Automatic Design Methods. pages 33–35, 2018.[39] Neil A Ernst, Rick Kazman, and Philip Bianco. Towards Rapid Composition with Confidence in RoboticsSoftware. pages 44–47, 2018.[40] Martin Molina, Abraham Carrera, Alberto Camporredondo, Hriday Bavle, and Alejandro Rodriguez-ramos.Building the executive system of autonomous aerial robots using the Aerostack open-source framework. (June):1–20, 2020.[41] Lorenzo Flückiger and Hans Utz. Service oriented robotic architecture for space robotics: Design, testing, andlessons learned.
Journal of Field Robotics , 31(1):176–191, 2014.[42] Gracian Trivino, Luis Mengual, and Albert van der Heide. Towards an architecture for semiautonomous robottelecontrol systems.
Information Sciences , 179(23):3973–3984, 2009.[43] Minseong Kim, Suntae Kim, Sooyong Park, Mun Taek Choi, Munsang Kim, and Hassan Gomaa. UML-based service robot software development: A case study.
Proceedings - International Conference on SoftwareEngineering , 2006(Icse):534–543, 2006.[44] Elena Messina, John Horst, Thomas Kramer, Hui Min Huang, and John Michaloski. Component specifications forrobotics integration.
Autonomous Robots , 6(3):247–264, 1999.11
PREPRINT - F
EBRUARY
26, 2021[45] Kim Minseong, Kim Suntae, Park Sooyong, Choi Mun-Taek, Kim Munsang, and Hassan Gomaa. Service robotfor the elderly: Software development with the COMET/UML method.
IEEE Robotics and Automation Magazine ,16(1):34–45, 2009.[46] Thomas Witte and Matthias Tichy. Checking consistency of robot software architectures in ROS.
Proceedings -International Conference on Software Engineering , pages 1–8, 2018.[47] Yoonseok Pyo, Kouhei Nakashima, Shunya Kuwahata, Ryo Kurazume, Tokuo Tsuji, Ken’ichi Morooka, andTsutomu Hasegawa. Service robot system with an informationally structured environment.
Robotics andAutonomous Systems , 74:148–165, 2015.[48] Moonzoo Kim, Jaejoon Lee, Kyo Chul Kang, Youngjin Hong, and Seokwon Bang. Re-engineering softwarearchitecture of home service robots: A case study.
Proceedings - 27th International Conference on SoftwareEngineering, ICSE05 , pages 505–513, 2005.[49] Nathan Michael, Jonathan Fink, and Vijay Kumar. Experimental Testbed for Large Multirobot Teams: Verificationand Validation.
IEEE Robotics and Automation Magazine , 15(1):53–61, 2008.[50] Daniel Castro Silva, Rodrigo A.M. Braga, Luís Paulo Reis, and Eugénio Oliveira. Designing a meta-model for ageneric robotic agent system using Gaia methodology.
Information Sciences , 195:190–210, 2012.[51] Pierre A. Akiki, Paul A. Akiki, Arosha K. Bandara, and Yijun Yu. EUD-MARS: End-user development ofmodel-driven adaptive robotics software systems.
Science of Computer Programming , 200:102534, 2020.[52] Alexandras Paraschos, Nikolaos I. Spanoudakis, and Michail G. Lagoudakis. Model-driven behavior specificationfor robotic teams. , 2(March 2014):696–703, 2012.[53] E. Coevoet, T. Morales-Bieze, F. Largilliere, Z. Zhang, M. Thieffry, M. Sanz-Lopez, B. Carrez, D. Marchal,O. Goury, J. Dequidt, and C. Duriez. Software toolkit for modeling, simulation, and control of soft robots.
Advanced Robotics , 31(22):1208–1224, 2017.[54] Diego Alonso C Aceres, Francisco J Ortiz, Juan Pastor Franco, and Pedro S Anchez. Design of Service Robots.(March), 2009.[55] Model-driven analysis and design for software development of autonomous underwater vehicles.
Robotica ,33(8):1731–1750, 2015.[56] G. T. McKee, J. A. Fryer, and P. S. Schenker. Object-oriented concepts for modular robotics systems. "Technologyof Object-Oriented Languages and Systems , (TOOL):229–239, 2001.[57] Politecnico Milano, Andrea Semprebon, Politecnico Milano, Matteo Matteucci, and Politecnico Milano. A usecase in model-based robot development using AADL and ROS Gianluca Bardaro. pages 9–16, 2018.[58] Loli BurguEo, Tanja Mayerhofer, Manuel Wimmer, and Antonio Vallecillo. Using physical quantities in robotsoftware models.
Proceedings - International Conference on Software Engineering , pages 23–28, 2018.[59] Youngdo Cho, Hwangwook Kim, Dae Kyoo Kim, and Sooyong Park. An interaction-driven approach toidentifying functional behaviors of service robot systems.
Proceedings - 2012 IEEE 17th International Conferenceon Engineering of Complex Computer Systems, ICECCS 2012 , pages 109–118, 2012.[60] Mahmoud Abdelgawad, Sterling McLeod, Anneliese Andrews, and Jing Xiao. Model-based testing of a real-timeadaptive motion planning system.
Advanced Robotics , 31(22):1159–1176, 2017.[61] Anneliese Von Mayrhauser and Stewart Crawford-Hines. Automated testing support for a robot tape library.
Proceedings - International Symposium on Software Reliability Engineering, ISSRE , pages 6–14, 1993.[62] Jennifer Carlson and Robin R. Murphy. Reliability analysis of mobile robots.
Proceedings - IEEE InternationalConference on Robotics and Automation , 1:274–281, 2003.[63] Afsoon Afzal, Claire Le Goues, Michael Hilton, and Christopher Steven Timperley. A Study on Challengesof Testing Robotic Systems.
Proceedings - 2020 IEEE 13th International Conference on Software Testing,Verification and Validation, ICST 2020 , pages 96–107, 2020.[64] R. E. D. Fairley, P. Bourque, and J. Keppler. The impact of swebok version 3 on software engineering educationand training. In , pages192–200, 2014.[65] Darko Bozhinoski, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, and Ivica Crnkovic. Safety for mobilerobotic system: A systematic mapping study from a software engineering perspective.