Cloud Broker: A Systematic Mapping Study
Hoda Taheri, Faeze Ramezani, Neda Mohammadi, Parisa Khoshdel, Bahareh Taghavi, Neda Khorasani, Saeid Abrishami, Abbas Rasoolzadegan
CCloud Broker: A Systematic Mapping Study
HODA TAHERI, FAEZE RAMEZANI, NEDA MOHAMMADI, PARISA KHOSHDEL, BAHAREHTAGHAVI, NEDA KHORASANI, SAEID ABRISHAMI ∗ , and ABBAS RASOOLZADEGAN, Com-puter Engineering Department, Ferdowsi University of Mashhad, Iran
In a cloud environment, a cloud broker is an important entity that works as an independent middleware between cloudcustomers and providers to address the issues related to satisfying the customer preferences and the service provider profitsthrough negotiation between them. In recent years, researchers have published many articles which are directly or indirectlyrelated to this research area. A systematic method is vital for extracting all search spaces (journals, conferences, and workshops)and primary studies (articles) conducted in the field of cloud broker and then selecting some studies with the highest quality.An important part of the presented systematic review is providing an appropriate research method that can extract largevolumes of related studies. The current systematic review includes a comprehensive 3-tier strategy (manual search, backwardsnowballing, and database search). The accuracy of the search methodology has been analyzed in terms of extracting relatedstudies and collecting comprehensive and complete information in a supplementary file. In the search methodology, qualitativecriteria have been defined to select studies with the highest quality and the most relevant among all search spaces. Also,some queries have been created using important keywords in the field under study in order to find studies related to thefield of the cloud broker. Out of 1928 extracted search spaces, 171 search spaces have been selected based on defined qualitycriteria. Then, 1298 studies have been extracted from the selected search spaces and have been selected 496 high-qualitypapers published in prestigious journals, conferences, and workshops that the advent of them has been from 2009 until theend of 2019. In Systematic Mapping Study (SMS), 8 research questions have been designed to achieve goals such as identifyingthe most important and hottest topics in the field of cloud broker, identifying existing trends and issues, identifying activeresearchers and countries in the cloud broker field, a variety of commonly used techniques in building cloud brokers, varietyof evaluation methods, the amount of research conducted in this field by year and place of publication and the identificationof the most important active search spaces. This information can provide a useful guide for research teams and developersinterested in this field.CCS Concepts: •
Computer systems organization → Architectures .Additional Key Words and Phrases: service selection, service composition, cloud broker, systematic review, systematic mappingstudy (SMS)
ACM Reference Format:
Hoda Taheri, Faeze Ramezani, Neda Mohammadi, Parisa Khoshdel, Bahareh Taghavi, Neda Khorasani, Saeid Abrishami,and Abbas Rasoolzadegan. 2021. Cloud Broker: A Systematic Mapping Study. 1, 1 (February 2021), 43 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn ∗ corresponding authorAuthors’ address: Hoda Taheri; Faeze Ramezani; Neda Mohammadi; Parisa Khoshdel; Bahareh Taghavi; Neda Khorasani; Saeid Abrishami;Abbas Rasoolzadegan, emails:[email protected], [email protected], [email protected], [email protected],[email protected], [email protected], [email protected], [email protected], Computer EngineeringDepartment, Ferdowsi University of Mashhad, Mashhad, Iran.Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided thatcopies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the firstpage. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copyotherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions [email protected].© 2021 Association for Computing Machinery.XXXX-XXXX/2021/2-ART $15.00https://doi.org/10.1145/nnnnnnn.nnnnnnn , Vol. 1, No. 1, Article . Publication date: February 2021. a r X i v : . [ c s . D C ] F e b • H. Taheri and F. Ramezani, et al. In the cloud environment, cloud services comply with pay-as-you-go logic that means each cloud customershould pay money as much as its consumption [22]. Cloud services have many benefits such as high availability,flexible application deployment, low cost etc., nevertheless, the assumption of cloud brokers is still in the infancystage [10, 13]. The market of cloud services consists of a huge number of services that many of them havethe same functionality with different quality. Therefore, the selection of proper services considering customerpreferences is a big challenge for cloud customers. A broker can play an important role in solving this challengethrough negotiation between all providers and cloud customers to find the most suitable services considering thecustomer preferences and the provider’s profits [13, 17].In some situations, cloud customers may become dependent on a particular cloud service provider that is knownas a lock-in problem and customers cannot easily move between cloud providers without paying an extra cost. Abroker can help cloud customers to avoid vendor lock-in problem. These benefits cause a lower cost in offeringservices and create a flawless switch between cloud providers to satisfy the customers’ preferences [4]. To actualizethe broker, applications should be able to remove the limitations of each cloud provider that led to providingthe cross-cloud computing [13] aimed at supporting developers versus challenges related to interoperability,migration, resource planning strategies, and dynamic deployment. Usually, the term "broker" has been utilized todepict various intermediation models. One of these models is a cloud federation that creates a common technologyto implement cloud services in which all providers should obey it. In contrast, a multi-cloud model does notconsider any common technology. Therefore, to switch between cloud providers, the broker should solve thedifferences between all providers [10]. Based on the important role of the broker, in the past decade, a largevolume of research study has been done on investigating different responsibilities of the broker.NIST has classified services offered by cloud broker in three categories namely: arbitration, aggregation andintermediation [28]. In cloud service aggregation, multiple cloud services have been combined and aggregatedinto one service. The broker is responsible to provide security of data when data is transferred between cloudcustomer and multiple cloud providers [28]. In aggregation, two services or more have been aggregated in singleservice to increase the broker capabilities [8]. Cloud service arbitrage is more flexible than service aggregation.Since in the service arbitration, the selection of service may be done from different providers. In other words, inservice arbitrage, the broker can select service from different providers based on the characteristics of the data orthe context of the service [28].However, there are a few studies to systematically investigate and analyze the area of cloud brokerage. Theexistence of a comprehensive and systematic review of the research in this field is crucial that can help to findthe major trends and issues. It can be used as a guideline for all researchers and enthusiasts to find a deeperunderstanding of the challenges and issues that need to be addressed. An accurate methodology is needed tocover and review all high-quality relevant research studies. This methodology should have some importantfeatures such as reliability, impartiality and also its results should be traceable. In 2005, Deba et al. [9] introducedan evidence-based software engineering method that consists of two well-known methodologies, i.e. systematicliterature review (SLR) and systematic mapping study (SMS) [26, 33]. To search all research works and reviewthem, both SMS and SLR have the same methodology [25, 26, 33]. However, Zhang and Budgen [26, 33, 39]illustrated some differences between the two methodologies. The major difference is in the determination of thefinal goal. Indeed, both methodologies have different research questions that should be answered at the end ofthe review. It can be stated that in SMS the research questions are more general and the goal is identifying theresearch trends and the topics in the specified field, while the SLR tries to extract the data from the initial studiesand subsequently answers some specified RQs [26]. As an important point, the field of security is a very broadarea that can be considered separately. For this reason, this SMS does not address articles related to the field ofsecurity, and such articles have been excluded during the search process for selecting related studies. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 3
In order to have proper coverage of the research studies conducted in the field of the cloud broker, a systematicmethodology is essential to detect all search spaces and research studies in this area. Hitherto, we have conductedsome SLR and SMS on the different research fields [20, 29, 36]. In 2017, a search methodology was done toreview studies on software design patterns [29] and then in next works [20, 36] the methodology was partiallyimproved. Ahmadian et al. used a comprehensive search methodology to review Intrusion alert analysis inintrusion detection systems [36], and in another review study Javan et al. conducted the review of researchstudies on security patterns in software design systematically [20]. As previously mentioned, a primary part ofthis methodology is presenting an appropriate search strategy to extract related research studies in the fieldunder study. The current systematic review is an extended version of our previous works, [36] and [20], andcomprises a 3-tier strategy i.e. manual search, backward snowballing, and database search.In presented search methodology, some qualitative criteria are defined to select the highest quality and mostrelevant articles among all search spaces and studies. Also, to find studies related to the field of the cloud broker,some queries have been designed based on important keywords in the field under study. The accuracy of thesearch methodology in finding related studies has been computed and also for more clarity, a supplementary filetitled
SuppFile is created which consists of some document files. The
SuppFile has all gathered data in SMS andpresents a comprehensive knowledge about all data in the desired filed. A quick guideline to use the
SuppFile is placed in http://sqlab.um.ac.ir/images/219/files/readme.pdf. During this paper, some data, tables and otherinformation from
SuppFile may be referred. For example,
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 refers to Table 1 of an excel file 𝐸
1, or
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 refers to Table 5 of a document 𝑊 SuppFile folder.In this Systematic Mapping Study (SMS), 8 research questions have been designed to achieve goals such asidentifying the most important and hottest topics in the field of cloud broker, identifying existing trends and issues,identifying active researchers and countries in the cloud broker field, a variety of commonly used techniques inbuilding cloud brokers, a variety of evaluation methods, the amount of research conducted by year and place ofpublication, and the identification of the most important active search spaces. This information can provide auseful guide for research teams and developers interested in the desired field. An SMS can be used as a pre-reviewbefore conducting SLR to gather general information on the desired research topic. As the starting point of theSMS on the cloud broker, as seen in Table 1, 24 secondary studies (survey and SLR) have been randomly selectedto investigate the literature relevant to the cloud broker. It should be noted that during conducting the researchmethodology, other review studies have been found that can be seen in the
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 .As previously mentioned, the presented search strategy has a 3-tier process including manual search, backwardsnowballing search and database search. In the manual search phase, each venue in the search space list (thathave been acquired from investigating references of existing reviews in Table 1) is manually searched using aset of constructed queries. Each query is comprised of a set of keywords that can be seen in the 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 .Table 2 illustrates the constructed queries to find related papers and studies.The goal of the backward snowballing phase is finding some new papers which have not been found in theprevious phase. In the snowballing phase, the references of all currently included papers are investigated. Itshould be noted that in each phase of finding new studies, the papers have been evaluated in terms of quality andjust a set of papers with desired quality have been selected and placed in the set of included papers. In Section 2,the criteria set to evaluate all new search spaces and new papers are explained in detail. In the database searchphase, a manual search is done using defined keywords in the 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 on well-known databases suchas Google Scholar, Springer Link, IEEEXplore, ACM Library, and ScienceDirect. By applying all search phasestogether, a large dataset of 1298 related papers in the field of the cloud broker from 2009 to the end of 2019 hasbeen extracted. Then, 496 papers have been selected according to some inclusion and exclusion criteria, andfinally, the selected papers are analyzed and have been used to answer 8 research questions. The details of theprocess have been described in the following sections. , Vol. 1, No. 1, Article . Publication date: February 2021. • H. Taheri and F. Ramezani, et al. Table 1. The Secondary Studies used for Generating Initial Set of our SMS
No. Secondary Study Title Year Ref.1
Brokering in Interconnected Cloud Computing Environments: A Survey 2018 [8] A Review on Service Broker Algorithm in Cloud Computing 2017 [21] A Comprehensive Study on Cloud Service Brokering Architecture 2017 [27] Cloud Services Recommendation Reviewing the Recent Advances and Suggestingthe Future Research Direction 2017 [5] Service Provisioning in Cloud: A Systematic Survey 2017 [7] A Survey on Various Cloud Aspects 2016 [35] A Classification and Comparison Framework for Cloud Service Brokerage Architec-tures 2016 [14] A Review on Broker Based Cloud Service Model 2016 [34] Cloud Service Brokerage: A Systematic Literature Review using a Software Develop-ment Lifecycle 2016 [32] Resource Provision Algorithms in Cloud Computing: A Survey 2016 [40] Towards a Holistic Multi-Cloud Brokerage System: Taxonomy, Survey and FutureDirections 2015 [3] A survey on SLA-based Brokering for Inter-Cloud Computing 2015 [31] Cloud Services Brokerage: A Survey and Research Roadmap 2015 [6] Cloud Service Selection: State-of-the-art on Future Research Directions 2014 [35] Cloud Computing Service Composition: A Systematic Literature Review 2014 [23] A Comparative Study of Traditional Cloud Service Providers and Cloud ServiceBrokers 2014 [24] A Review of Literature on Cloud Brokerage Services 2014 [2] A Literature Review on Cloud Computing Adoption Issues in Enterprises 2014 [11] A Survey on Needs and Issues of Cloud Broker for Cloud Environment 2014 [16] Survey on important Cloud Service Provider attributes using SMI Framework 2013 [30] A Comparison Framework and Review of Service Brokerage Solutions for CloudArchitectures 2013 [15] A Survey on Interoperability in the Cloud Computing Environments 2013 [37] A Survey on Infrastructure Platform Issues in Cloud Computing 2012 [1] Inter-Cloud Architectures and Application Brokering: Taxonomy and Survey 2012 [18]
Table 2. Constructed queries for finding the related studies
Cloud broker Cloud AND service (arbitration OR intermediation OR aggregation OR integration OR cus-tomization OR Orchestrat) (Multi cloud OR Federat cloud OR Cross cloud OR Inter cloud OR (third party AND Cloud))AND (auction OR negotiation OR pricing OR interoperability OR management) Cloud AND ("service composition" OR "service selection") , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 5
This paper is structured as follows. Section 2 presents the research methodology used in this SMS. Section 3presents the results of the current systematic study that has been categorized by 8 research questions. In Section 4,the acquired results of Section 3 have been discussed. In Section 5, some implications based on the results of thecurrent SMS are presented for researchers, stakeholders and practitioners, educators and teachers interested inthe field under study. Finally, SMS in Section 6 has been concluded.
Already, distinct works have been done for conducting SMS and designing a unique research methodology [14, 34].One of the most prominent of these research methodologies is related to Peterson et al. [14]. The research method-ology used in this paper is adapted from three SMSs that were done by Peterson et al. [14], Ramaki et al. [36],and Javan et al. [20] and some updates and improvements in some phases have been done. It’s worth mentioning,considering the investigation of research studies, the advent of the cloud broker is 2009. Therefore, this SMScovers all published research articles from 2009 until end of 2019 and a complete review have been conducted onthe cloud broker research works during this period. Generally, the suggested SMS consists of two main steps, i.e.,planning step and conducting step. In the following, due to space constraint, each of these steps are describedbriefly and in Appendix A has been completely explained.The planning step comprises seven fundamental phases. The first phase is specifying the scope and researchquestions. The goal of defining research questions is determining the research goals that during conductingthe SMS are responded to them. In this SMS, 8 comprehensive and different RQs have been defined, whichresponding them can cover all our objectives. Table 3 describes these RQs and explains the rationality of eachof them. The second phase is specifying the search strategy. In this phase three search strategies, i.e., databasesearch, snowballing search, and manual search is applied to find related studies.The third phase is to specify the search space. In fact, search spaces are publishers who have published studies inthe field of the cloud broker. At the beginning of the review process, the search space set is empty. Therefore,as described in Section 1, a set of secondary studies (Table 1) are selected to begin the review process. The listof search spaces is quantified by reviewing the cited articles in the references section of the secondary studies.The fourth phase is to specify the search string. After quantifying the search space list, it is necessary to startsearching for studies related to the broker field in the search spaces through search strings. Therefore, at thisstage, related keywords are merged and queries are created. These queries, which are the same as search strings,are used to search for related studies in search spaces. Table 2 shows the queries used in this SMS.The fifth phase is planning the study selection process. After finding the studies related to the broker field throughthe previous phases, among the related studies, studies that have the necessary qualifications will be reviewedand analyzed. For this purpose, a set of quality criteria has been defined, according to which studies published injournals, conferences and workshops are qualitatively reviewed and selected. Details and tables related to thesequality criteria can be seen in Appendix A. Tables 12, 13 and 14 illustrate the exclusion criteria for journals, theexclusion criteria for conferences and workshops and the exclusion criteria of the extracted studies, respectively.A full description of each of these criteria can be found in Appendix A.Complete list about all extracted search spaces and reason for exclusion and inclusion of journals, and other(conferences, and workshops) have been provided in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 respectively. 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 present list of extracted papers and comprehensive information about thereason for the exclusion of each extracted paper. The main aims and scope of the cloud broker field have beenspecified in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 . Moreover, the aims and scope of each of the search spaces have been introduced in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 .After determining the included studies, by investigating the keywords of the included papers, some new keywords , Vol. 1, No. 1, Article . Publication date: February 2021. • H. Taheri and F. Ramezani, et al. Table 3. Defined Research Questions What are the core research topics in the field ofbrokering? To identify and classify the current research regardingbrokering techniques , analyze the evolution and distri-bution of each topic and the potential trends in the focusof researchers. Which broker topics have the least/most corre-sponding attention and what is the publicationtrend and distribution for each topic? Some objectives might be more prominent than others,but broker developers should take care to cover a variedspectrum of topics. Which forms of empirical evaluation have beenused? What are the tools available to support fieldapproaches? Which techniques are more used inthe field? The empirical evaluation means whether the environ-ment is real or simulation and supporting tools can de-scribe frameworks, platforms, or simulation. Techniquescan be game theory, optimization, and heuristic. What is the relation between topics and brokerroles in NIST category? Which NIST roles havethe least/most corresponding attention? General classification schemes might work to an extent,but a precise and comprehensive classification of brokerroles should address broker-specific criteria. In which environment and service layer is thebroker mostly considered? Environments are Multi-cloud, federated, etc. and theservice layer is IaaS, SaaS, PaaS, and XaaS. What is the broker Control orientation? Generally, types of control orientation are centralizedand distributed.
Table 4. Journal Exclusion Criteria (JEC)
If the journal is not indexed in the JCR If the scope of the journal is not related to our desired field
Table 5. Other Exclusion Criteria (OEC) ((Qualis < A5) OR (ERA < A) OR [(H5_Index <
15) AND ((Qualis < A5) OR (ERA < A))] OR (Metrics Not Available)) (Aims and scopes are not related)are extracted and added to the set of keywords for later use. 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 , ( 𝑇 𝑡𝑜𝑇 ) illustrate com-plete information of all other papers (conferences and workshops) and journal papers respectively. 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 presents a complete list of all included papers (journals, conferences, and workshops). It should be noted thatthere are some valuable points that are explained below: , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 7 Table 6. Exclusion criteria for extracted studies
The study is not a primary study (e.g. survey) Study cannot be accessed (e.g. not indexed) The study is out of our primary scope( e.g. security) The study belongs to an excluded search space The contribution of the study is not related to the cloud broker. • The field of cloud broker comprises a large pool of research studies, therefore some thresholds for theexclusion criteria considering two principals have been empirically selected: 1) a small change in theexclusion thresholds should not have a big effect on the number of excluded/included papers and 2)applying these thresholds should not exclude a lot of highly cited papers. • Although the desired field in this SMS is cloud broker, but search spaces with the scope and aim of webservices or distributed and parallel computing have been considered. There are also search spaces in otherresearch areas which are included, because they have published some papers in the field of cloud computing.An example is IEEE Transaction on Smart Grid. • There are some search spaces such as the International Conference on Software Engineering (ICSE) that havescarcely published papers in the field of cloud computing. Because its scope is about software engineering.Therefore, such search spaces are excluded. • In rare cases, we found a search space with two names. For example, the ACM International Symposium onHigh-Performance Distributed Computing and International Symposium on High-Performance Parallel andDistributed Computing are two names for the search space of HPDC. Also, there is a search space that itsname had been changed. The initial name for GLOBECOM search spaces is IEEE Global TelecommunicationsConference, and after 1972, its name has been changed to Global Communications Conference. As anotherexample, after 2007, two search spaces namely, European Conference on Machine Learning (ECML) andEuropean Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD) have beenmerged titled European Conference on Principles and Practice of Knowledge Discovery in Databases(PKDD).The sixth phase is specifying the search and study evaluation strategy. In this phase, the goal is to examinethe completeness of the search strategies used to find related studies. Therefore, objective evaluations (i.e.,quantitative criteria) and subjective evaluations (managed by an expert) have been used to evaluate searchstrategies. In this review, two prominent and most valid metrics are used to evaluate the search strategy. In otherwords, both objective evaluations (i.e. quantitative criteria) and subjective evaluation (managed by expert review)are done using Eq. 3 and Eq. 4 respectively. To have a more objective evaluation, the quasi-sensitivity metric isused to evaluate the applied search and study selection.
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = The Number of Studies in Our SMSThe Number of Studies Overall ×
100 (1)
𝑄𝐺𝑆 = number of discovered papers in the search phasenumber of discovered papres in the evaluation phase ×
100 (2)QGS alludes to a set of studies published in the well-known research communities. To create this gold standard,the home pages of active researchers in the area of cloud computing have been visited and their papers in the cloudbroker field have been extracted. After extracting studies from home pages and applying the inclusion/exclusioncriteria, the remaining unseen papers comprise our GCS according to Eq. 4. The aim of applying the QGS is , Vol. 1, No. 1, Article . Publication date: February 2021. • H. Taheri and F. Ramezani, et al.
Main Topics in Cloud Broker Pricing 4.23 %Client Centric 70.2 %Miscellaneous 1.81% Other
Cloud 12.7 %Intercloud 11.69%
Provider-Centric 31.65%
IaaS 7.66 % Other 1 %XaaS 6.65 %SaaS 5.44 %Service Selection 19.95 %Service Discovery
Service Allocation
Recommendation 4.83%Service Composition and Integration 26.61%
Energy Management
SLA Management
Resource Allocation
Fig. 1. Research Tree calculating the quasi-sensitivity and then comparing the obtained result with a predefined threshold. Accordingly,if the result falls below the threshold, the search and study selection process should be repeated using QGS. Byfollowing [5], an acceptable threshold should be between 70% and 80%. In the evaluation phase, 32 articles werefound, of which 26 articles were found in the main search phases of the systematic review process, therefore, weachieved a sensitivity of 81.25% which is above our predefined threshold. It means that the probability of notfinding a paper related to the cloud broker is less than 20%. Therefore, it can be concluded that the acquiredresults from this review have enough accuracy and validity.Details of the steps taken to evaluate the search strategies used and the metrics used to calculate the accuracyand completeness of these strategies to extract related studies can be found in Section 1.6 of Appendix A.The seventh phase is planning the data extraction and classification process. After determining the includestudies based on the defined quality criteria, information is extracted from this collection of included papers toanswer research questions and this information is stored in tables. To extract such information, different partsof the article such as title, abstract, keywords and body of the article are examined. After organizing data intotables, to respond to the RQs of the SMS, these data should accurately be investigated and analyzed. The mostimportant objective in the presented SMS is determining the primary topics and sub-topics in the cloud brokerfield.
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ( , 𝑡𝑜 ) include comprehensively all needed information to answering RQs.To find important topics in the broker field, keywords are extracted from included studies and aggregated andclassified considering their semantics. In this SMS, the classification and aggregation of the extracted keywordshave been done in 18 steps, which will eventually lead to the creation of a research tree shown in Figure 1. Aresearch tree is a multilevel tree that contains the main topics and sub-topics in the desired field. The mostimportant level in this tree is its root. Because all analysis to respond to RQs is done based on the located topicsand sub-topics into the root level. The importance of each keyword is determined by the amount of its repetitionin the included studies. Section 3.4 in Appendix A provides a complete description to find important topics andto determine the steps for building the research tree. The following is a brief description of the steps of building aresearch tree.Once the list of included studies is complete, in the first step of building a research tree, the keywords of thestudies are extracted. If the study has not any keyword, the content of the study is reviewed by an expert andsome keywords are determined for it. After extracting keywords from all included studies, the repetition of each , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 9 word in the total included studies is calculated. Keywords with a repetition greater than or equal to 9 are storedin a co-occurrence matrix. The second step is to integrate related keywords that are semantically close to eachother. The goal is to determine a specific topic for a group of keywords. For example, service composition andintegration include concepts such as composition, orchestration, and integration. Therefore, the keywords relatedto these three categories are part of the topic of service composition and integration illustrated in Figure 1.The integration of related keywords is done in several steps which can be found in Appendix A. By examining therelated topics by experts and aggregating them, finally 10 main topics have been created in the cloud broker fieldthat is introduced as the level-one topics of the research tree. Topics are divided into two categories: client-centricand provider-centric. Client-Centric topics are activities that the broker performs as a result of the user request.For example, as a result of a service request by a user, the broker may perform service discovery, service selection,service composition, and so on. Provider-Centric topics are activities that the broker performs as a result of theprovider’s request. For example, pricing, resource allocation, energy management, etc. It should be noted thatservice allocation includes service provisioning and scheduling on the client-side. Figure 1 in article shows theresearch tree obtained in this SMS. The percentage of included papers in each topic is shown below the topic.One of the rubrics applied for evaluation of our SMS is threats to validity. In the validation process, the primarygoal is providing some evidence for resolving all existing threats facing our SMS. In the following, the prominentevidence is investigated and discussed. • Obtaining a set of high-quality studies:
For obtaining a complete set of high-quality studies in the fieldof the cloud broker, a complete procedure is designed as a search strategy that comprises advantages ofboth famous search methods i.e. SLR and SMS. Therefore, we believe that our review is reliable. • Obtaining the most related studies:
One of the most primary advantages in our search strategy isgradual evolution of keywords set during conducting the SMS. In should be stated that in some situation,some keywords in the set of keywords did not convey a concept of the cloud broker. Accordingly, all suchkeywords in a category by applying some logical operators (AND, OR) have been merged. Examples ofmerging some keywords are provided in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 . • Reviewer’s biases or misunderstandings during the process of study selection:
To prevent thesechallenges in our SMS, at first, the selection process is independently done by two reviewers, then thepossible disagreements were solved by the third reviewer and decision rules. • Creating some forms to extract raw-data:
As another threat, during the execution of the data extractionphase, some included studies were without author’s keywords and we extracted suitable keywords forthose studies considering the context of it and stored the extracted keywords in the related forms. • Assigning proper name for each level-one topic:
After clustering all keywords (extracted and gener-ated), we assign a proper name for each cluster that describes the concept of that cluster. For example, forthe
Resource Allocation topic, there were some keywords like
Resource Allocation , Resource Management ,and
VM Scheduling . However, we pick out
Resource Allocation as a suitable name for the topic, because webelieve this name can goodly convey the desired concept of studies. Nevertheless, in the presented SMS,in addition to both considering semantic and syntax of keywords for each topic, we also brought up theselected names between our team to remove the potential bias.
In the previous sections, we explained the process of searching for study and space in detail. In this section, weanalyze and respond to the presented RQs in Section 2 based on their arrangement in Table 3. It is worth notingthat level-one topics in research tree cover all sub-topics at lower levels. For this reason, the presented analysis iscarried out on these topics. , Vol. 1, No. 1, Article . Publication date: February 2021.
Fig. 2. Number of included papers per year
One of the primary RQs is investigating the frequency of published papers in the field of cloud from the beginningof the advent of brokers until the end of 2019. Considering the studies extracted by the queries defined in Table 2,we observed that the time interval of published studies in the field of brokers is from 2009 to 2019. This analysisdemonstrates the level of acceptance and progress of the research field of broker in the period under review. Weillustrate the level of attention of academic societies and its changes over these years. By analyzing the includedstudies set, a logical trend in published studies has been found that focused on the responsibilities of brokers inthe cloud environment. The final results is illustrated in Figure 2. The information depicted in Figure 2 to Figure 5are acquired based on the analysis of inserted information in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 , 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 ( 𝐸 𝑡𝑜𝐸 ) . As can be seen inFigure 7, among 496 included studies, 55% of them were published between 2017-2019. The number of papers inthis field has increased since 2012 with the advent of the multi-cloud concept and has intensified in recent years,and in 2019 the broker field has had the highest number of studies compared to previous years. Figure 3 showsthe frequency of top-level topics in the included studies between 2009-2019. As can be seen, some topics have ahigher frequency in studies. Service Composition and Integration, Service Selection, and Service Allocation havethe most frequency in the included studies. The reason for more paying more attention to the concepts of servicecomposition and service selection is the complexity of these topic. In general, topics that were scientifically morecomplex are more popular. Moreover, these topics are the main services of a cloud broker according to NISTdefinition. Topics such as service discovery and monitoring are more technical and less complex, and there arefewer papers on these concepts. Besides, some topics such as pricing and recommendation have a good potentialto be investigated in the future.From another point of view, we can examine the importance of the topics extracted considering the queriesutilized in the search phase for finding related studies. Figure 4 shows the importance of the queries defined inTable 2 in retrieving related papers and states that the queries are well designed and have an equal contribution(approximately 25%) in the results.It is worth noting that of the 496 included studies, 23% of the studies (115 studies) were retrieved throughthe first query, of which 21% (101 studies) were extracted directly using the first query and 2% of the remained , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 11 Fig. 3. Published Papers per Topic per YearFig. 4. Included paper per query studies have been extracted using the combination of the first query with other queries. The highest overlap wasbetween queries 1 and 3 with 8 included studies. According to observations, queries 1 and 3 have the highestoverlap. Another type of analysis of the included studies is based on search spaces. Among 496 extracted studies, , Vol. 1, No. 1, Article . Publication date: February 2021.
Table 7. The most important journals in the field of cloud broker
Journal Title Number of Studies
Future Generation Computer Systems 80IEEE Transactions on Services Computing 29IEEE Transactions on Cloud Computing 29IEEE Transactions on Parallel and Distributed Systems 21Journal of Network and Computer Applications 18The Journal of Supercomputing 15Cluster Computing 15Journal of Systems and Software 11Journal of Grid Computing 10Concurrency and Computation: Practice and Experience 8KSII Transactions on Internet and Information Systems 7ACM Transactions on Internet Technology 6Journal of Parallel and Distributed Computing 5International Journal of Computer Integrated Manufacturing 5
Table 8. The most important conference in the field of cloud broker
Conference Title Number of Studies
International Conference on Service Oriented Computing (ICSOC) 25International Conference on Utility and Cloud Computing (UCC) 19International Conference on Web Services (ICWS) 16International Conference on Cloud Networking (CloudNet) 13International Conference on Cloud Engineering 11ACM/IFIP/USENIX International Middleware Conference (Middleware) 9International Conference on Future Internet of Things and Cloud 7International Conference on Distributed Computing Systems (ICDCS) 6International Conference on Big Data (BigData) 6IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID) 6International Conference on Computing, Networking and Communication (ICNC) 5326 studies were published in journals and 170 studies were published in conferences and workshops. Table 7and Table 8 demonstrate the most important journals and conferences respectively.Figure 5 shows the comprehensiveness of queries. Table 9 shows the number of Included and Excluded papersin different phases. As previously mentioned, our SMS process consists of three main phases i.e. initial searchphase (extraction of search spaces set and the studies set found from the initial set), snowballing search phase,and evaluation phase. Table 9 provides complete information and shows the number of included and excludedstudies in each phase. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 13
Fig. 5. Total extracted studies per each search phaseTable 9. The number of Included and Excluded papers in different phases
Search Space Initial Set Manual & Snowballing Database Search-TestIncluded Excluded Included Excluded Included Excluded
Journals 284 378 42 37 25 95Others 133 279 37 108 5 13Total 417 657 79 137 30 108
Knowing the active researchers in the field of the cloud broker is useful for those who want to research andwork in this field. We have presented the acquired results of the evaluation phase in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 that have beenused to extract the active researchers. A list of active researchers can be seen in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 and the countryof authors can be found in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 , ( 𝑇 𝑎𝑛𝑑𝑇 ) . Figure 6 demonstrates the active researchers considering thenumber of publications in the field of the cloud broker. Researchers rank in descending order based on the numberof publications from left to right. Rajkumar Buyya with 22 publication in the field of the cloud broker is placedat the top of the list. It should be noted that the more publications an author have in each topic, the bigger thebubble size.Another type of analysis in our SMS is investigating the number of publications from the viewpoint ofgeographical distribution. This analysis can help us in finding the most important institutes and countries thathave a significant impact on advancing the cloud broker field. To conduct this analysis, the data obtained from theextraction phase is used. Complete information about the final included studies set can be found in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 .Since a study may have multiple authors with different affiliations, all affiliations will be considered in Geographicdistribution. To answer the geographical distribution RQ, the author affiliation in included papers are consideredand accordingly the frequency of each country is calculated. , Vol. 1, No. 1, Article . Publication date: February 2021. Fig. 6. The active researchers considering the number of publications in the field of the cloud broker
Obviously, in papers with more than one author, the country associated with each author is given in thecalculations depending on its affiliation. Figure 7 shows the geographical distribution of publications in the field ofresearch. A segment of this figure is named "Others" that aggregates the frequency of all countries with less than50 publications. According to the presented information in Figure 7, China has the largest share (289 publications)of total publications. Afterwards, the United States is ranked next in order. Researchers in these countries havepaid more attention to the field under study, and this may be due to the availability of the necessary platforms touse the cloud and brokerage infrastructure in research projects and implementation. China, the United States,and Australia have conducted more researches. Because they are richer and more technologically advanced. Inaddition, due to the existence of suitable infrastructures such as cloud data centers, the field of cloud broker inthese countries has received more attention. The existence of research laboratories related to cloud computing inuniversities such as the CLOUDS laboratory (managed by Rajkumar Buyya) has also been influential.
Main research topics are found by conducting topic detection process using clustering keywords. In Addition, weapply a procedure to reconstruct research-tree and identification of sub-topics in the cloud broker field. Thisprocedure can be seen in Appendix A, Section 3.4. The acquired research tree is illustrated in Figure 1 andcomprises 10 main topics as level-one topics. Furthermore, Table 10 in Appendix A demonstrate some extractedkeywords that have a primary role in identifying level-one topics. A comprehensive list of keywords for eachlevel-one topics can be seen in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 . In the following of this Section, each level-one topic is described indetail.(1) Service Discovery.
By using this technology, detecting cloud services and offering appropriate resourcesprovided by providers are automatically done on the internet.(2)
Service composition and integration.
Service composition and integration concern the value-addedservices and satisfy the demands of users. A cloud broker gathers all essential services considering the typeof user demand. Usually, it is possible that all services have been offered by just one provider, or sometimesit needs to combine different services of different providers. Therefore, a service composition procedure , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 15
Fig. 7. The geographical distribution of publications begins with the request of a complex service from the user, then a cloud broker finds and combines allappropriate services according to quality of services (QoS).(3)
Service allocation.
In general, the processes involved in providing the services intended by the userand/or scheduling tasks on virtual machines fall into this category. A cloud broker can help to optimize theallocation of tasks on virtual machines by providing scheduling services. All of the above concepts are alsocommon in inter-cloud environments.(4)
Energy management.
Energy management for a cloud provider reduces energy consumption and pro-duces less heat and carbon footprint. The cloud broker can help for optimization of energy management byproviding appropriate suggestions for running virtual machines on physical machines owned by one ormore providers.(5)
Service selection.
Optimal cloud deployment requires an effective selection strategy that operates accord-ing to QoS metrics such as cost, reliability, and security, and also offers the most appropriate cloud servicesfor end customers.(6)
SLA management.
Service-level agreements management is one of the challenges in cloud applications.With the advent of sophisticated applications that sometimes lead to the provision of services by severalcloud providers, coordination between service-level contracts and inter-cloud negotiations to provide thedesired QoS is crucial, which is provided by a third-party cloud broker.(7)
Resource allocation.
Resource management is one of the most important issues for a cloud provider. Thisrole includes managing virtual resources on the physical resources of cloud data centers (providing thephysical resource and allocating it to the virtual resource) as well as managing other resources such asstorage space and network resources. The cloud broker manages cloud resources as a third party or as apart of the cloud provider. Cloud resource management in a multi-cloud environment can also be done bymediation of a broker.(8)
Pricing.
This concept includes marketing-related mechanisms and pricing of resources and services ofone or more cloud providers. The broker can be involved in processes related to the cloud services market,such as holding auctions, defining new service buying and selling models, and offering profits to providers. , Vol. 1, No. 1, Article . Publication date: February 2021.
Fig. 8. Percentage of Publications per Topic (9)
Monitoring.
Cloud monitoring is a wide term that monitors diverse aspects of services, from VM per-formance to very complex monitoring of cloud services. It should be noted that, monitoring systems areneeded to monitor the performance of physical and virtual resources and to run cloud applications.(10)
Recommendation.
The cloud broker can detect and suggest appropriate services according to the qualityof the desired service of the user. It helps users choose the right service by offering the right offers. Aservice can also include data management mechanisms.
To answer the first part of RQ4, as shown in Figure 8, we compute the percentage of publication per topic. Thisfigure demonstrates the quota of each level-one topic in all publications in the field of the cloud broker. Ascan be seen, the
Composition and Integration topic has attracted the most attention. Afterwards, the
Selection and
Service Allocation topics are the second and third most important research topics in the cloud broker field,respectively. The level of researchers’ reception of research topics containing Discovery, Recommendation,Energy Management, and Monitoring are almost equal. Among all extracted level-one topics, topics of the SLAManagement and Pricing have the lowest publication rate.In confirmation of the information obtained from Figure 8, Figure 9 shows the evolution of each topic overtime so that among all the extracted level-one topics,
Composition and Integration and
Selection have been themost popular. In cloud computing, some services such as service allocation, service composition, and serviceselection are known as the most basic duties of a cloud broker. Accordingly, these topics attract more attention inresearch works. Furthermore, there are some other important topics such as discovery and monitoring serviceswhich for implementing them, complex algorithms are not needed. Hence, the conducted research works onthese topics are less than the first group. The third group are topics such as pricing and recommendation. Theseservices are higher level services and are expected to be of more interest to broker systems researchers in the , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 17
Fig. 9. Evolution publication of research studies in the client-centric topicsFig. 10. Evolution publication of research studies in the provider-centric topics future. In addition, the field of energy management is an area that is of great importance and has so far receivedless attention from researchers. We are expected to see new work in this area in the future. Evolution of researchstudies in both perspectives i.e. client-centric and provider-centric is shown in Figure 9 and Figure 10. , Vol. 1, No. 1, Article . Publication date: February 2021.
Fig. 11. Empirical Evaluation
Another valuable information extracted from this SMS is investigating the popularity of evaluation methods inthe cloud broker field. Empirical evaluation includes Test bed, Practical, Simulation. Testbed is an implementationof a real cloud on a smaller scale using cloud management software such as OpenStack. Practical is a realimplementation in a commercial cloud, which may also be referred to in articles as implementation. Simulation isa cloud simulator such as CloudSim or is a self-development wherein a problem is solved by existing programminglanguages such as Java, Python. Figure 11 demonstrates the types of evaluation methods used in the reviewedstudies.On the other hand, all presented and used techniques in the included studies have been considered. Therefore,by answering this RQ, researchers interested in the cloud broker field will become familiar with all types oftechniques used in this field to fulfil the demands of users. Most studies have proposed an architecture orframework, and after that heuristic and meta-heuristic algorithms have the next rank in search studies. Figure 12shows the techniques used in the studies to solve the existed issues in cloud environments by broker.
As can be seen in Figure 13, out of 496 included studies, 466 studies are related to categories of NIST that themajority of which (70%) have paid special attention to the role of intermediation. Because intermediation isthe simplest and most currently operational type of broker, while aggregation and arbitrage are relatively newconcepts. Also, intermediation is the primary type of broker and performs the most basic task of the broker, , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 19
Fig. 12. The used techniques to implement broker in the included studiesFig. 13. Relation between extracted topics and NIST roles which is to find the service. However, we expect in the future, with the advancement of broker capabilities, theconcepts of arbitrage and aggregation have been further explored as broker’s duties, and with the addition ofsuch a feature, brokers can combine different services to meet complex requests. Therefore, in the analysis of2009-2019 studies, 79 and 55 studies have arbitrage and aggregation as new capabilities of the broker, respectively.As described in Section 2, by examining the included studies, a set of important topics in the broker field(level-one topics in Figure 1) have been extracted. The purpose of answering this research question is to investigatethe relationship between the topics extracted with the role of broker, which is classified by NIST into three generalcategories aggregation, arbitrage, and intermediation. As excepted, Figure 14 demonstrates that the number of , Vol. 1, No. 1, Article . Publication date: February 2021.
Fig. 14. The frequency of roles of NIST in the included papers included papers which have considered the topic of composition and integration have paid close attention to theroles of aggregation and arbitrage and also among other topics, due to the more prominent role of intermediation,most researches have dealt with this topic.
One of the vital analyzes in the systematic review of the broker field is to pay attention to the level at whichthe broker plays a role. Based on the analysis of the studies, we have divided the levels of cloud services into5 categories SaaS, PaaS, IaaS (IaaS and CaaS), XaaS (all: anything as a service), and Other (special services e.g.NaaS (Network as a Service), DaaS (Desktop as a Service)). Since most commercial cloud services are in the IaaSlayers, the illustrated result in Figure 15 is predictable and also acceptable. It should be noted that if the type ofcloud services in included studies has not been explicitly indicated, we have considered the service as XaaS thatgenerally covers any type of cloud service.
In this RQ, we have analyzed the broker in the viewpoint of centralized or distributed orientation. In studies,broker implementation is divided into two categories, distributed and centralized. In the centralized model, asingle broker entity is responsible for managing the broker’s tasks, while in the distributed model, a number ofbrokers perform brokerage tasks in coordination with each other. It is clear that the centralized model is easier toimplement, and since all information is stored in the centralized broker, the decision making is easier. But thedisadvantage of this model is that the participants in the brokerage process may not be willing to provide theirinformation to the broker or it may be difficult for a centralized broker to have all the information. Figure 16,depicts types of cloud brokers. AS can be seen in Figure 16, a large number of the included studies have usedcentralized algorithms to present and demonstrate cloud broker capabilities. Accurate and instantaneous datacollection is difficult for a centralized broker, and some participants in the system such as providers in a federationmay be reluctant to share their information with a centralized broker. In this case, the distributed model is , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 21
Fig. 15. Types of service layer in the cloud computingFig. 16. Types of broker important and each provider has its own broker and the privacy of the providers is maintained, but each providermust estimate the information of other system members and make decisions based on its local data and just asmall piece of the information of other providers.
In this section, the SMS provided will be compared with other review articles in the field and the degree ofcompleteness of each of them will be discussed. The following sections will further review and analyze the results , Vol. 1, No. 1, Article . Publication date: February 2021. of Section 3, i.e. the types of evaluation methods used in the field of broker, cloud broker applications and typesof methods used for broker development.
The main objective to answer this RQ is providing a comparison between our SMS and other related reviews.As mentioned previously in Section 1, we have selected three review papers [19, 23, 32] from Table 1 that aremost similar to our SMS and then an exact comparison has been done between them. Table 10 illustrates thecomparison between our SMS with other three reviews. As seen, we investigate a large set of search spaces and thehigh-quality related papers are selected among a large pool of extracted papers. Also, we present a comprehensivesearch strategy which can find the majority of related search spaces and studies and then include/exclude papersconsidering some exclusion criteria presented in Appendix that was described briefly in Section 2.We perceived that all the investigated secondary works are placed in the range of 2012 until 2019. However, mostof them lack a systematic methodology to cover all relevant studies in the field of the cloud broker. Among the24 secondary articles, we selected three of them for more deeply investigation because they seem closer to thedesired fields and are more comprehensive. The selected surveys are Fowley et al. [23], Chauhan and et al. [32],and Elhabbash and et al. [19].As can be deduced from Table 10, our SMS (last row) investigate a large set of related papers comparing toother review studies. Most review studies have not a systematic process to review studies in the field of the cloudbroker and each of them just has investigated a small set of papers in the desired filed. In contrast, our searchstrategy has a comprehensive process in finding a complete set of related papers and select a set of high-qualitypapers in the field of the cloud broker. Because, our search strategy is a 3-tier process comprise manual search,snowballing search and database search. In the manual search phase, each venue in the search space list (thathave been acquired from investigating references of existed reviews in Table 1) is manually searched using aset of constructed queries. Each query is comprised of a set of keywords that can be seen in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 . , Asanother comparison, Table 9 illustrates a comparison between our SMS with more reviews from viewpoint of thepresented topics. Observing the demographics of the cloud broker research reveals the importance of broker in the researchconducted on the cloud environments. The upward growth of bar charts in Figure 2 (RQ1) conveys that cloudbroker has been widely accepted as one of the promising solutions in the cloud environments. We have identifiedkey researchers and research venues in the field of cloud broker to be introduced as a guideline for those whowant to research this area.A complete list of research venues has been placed in the
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 , ( 𝑇 𝑎𝑛𝑑𝑇 ) . It should be noted that knowingthe geographical distribution of publications shown in Figure 7 can help to find important geographical locationsfor cloud broker researchers. There is a direct relationship between the volume of research conducted in the fieldof the broker and the developed countries. Because in the developed countries, there are researchers as well astop research institutes such as CLOUDS laboratory of Buyya that are active in this field.Another valuable achievement for guiding broker enthusiasts is introducing the most important research topicsin the under-review field to academic researchers and industry specialists. According to the results obtained fromFigure 3, it is observed that the service composition, service selection, and resource allocation topics have thehighest amount of research and publication compared to other topics of broker until 2019. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 23 Table 10. Comparison between our SMS and other reviews
Ref. ReviewType
AggregationCustomization [33]
Survey NM 17 IntermediationIntegrationArbitragePricingMulti-Criteria [38]
Survey 47 Quality of ServicesOptimizationTrustDecision SupportResource MonitoringPolicy Enforcement [39]
Survey 34 30 2010 to 2017 SLA NegotiationApplication DeploymentMigrationAPI AbstractionVM InteroperabilityService Composition & IntegrationService DiscoveryService SelectionEnergy Management
Our SMS
SMS 496 171 2006 to 2019 SLA ManagementResource AllocationPricingRecommendationService AllocationMonitoring These values are not mentioned directly in the papers and are extracted manuallyWith the spread of startups, the need to get cloud resources is increasing. Using a resource allocation strategywill help providers for better use of their cloud resources and get the most revenue from them. Recently, withincreasing demand for cloud resources, research works have been more inclined to examine resource allocationalgorithms. On the other hand, with the advent of multi-cloud and inter-cloud environments, as well as thecreation of federations, the concept of combining cloud services and selecting the most suitable service providedby different providers have become particularly important. Figure 3 demonstrates that a significant amount ofresearch works has been done to introduce effective solutions on composition and integration, selection, andservice allocation topics.In cloud environments, the allocation of resources is elastic and virtual, which is provided in the web platformto meet the needs of cloud customers. With the expansion of cloud environments, many commercial companiescompete to sell their cloud resources and to attract customers. Accordingly, finding the most suitable service , Vol. 1, No. 1, Article . Publication date: February 2021.
Table 11. Comparison between our SMS and related reviews considering the extracted level-one topics
References [12] [14] [8] [23] [34] [18] [38] OurSMSTopicsComposition &Integration √ √ √ √
Discovery √ √ √
Service Allocation √ √ √ √ √
Energy Management √ Selection √ √ √ √
Resource Allocation √ √ √ √
Pricing √ √ √ √ √
Monitoring √ √ √ √
Recommendation √ √
SLA Management √ √ √ √ √ tailored to the needs of users among the large volume of cloud services is one of the important challenges that asone of the important tasks of brokers has found a special place. Considering the Acquired results from Figure 9and Figure 10, 15% of the included studies have suggested solutions to select the appropriate service, taking intoaccount the preferences of cloud customers. Figure 13 illustrates that 101 studies have been conducted on theselection of cloud service in which the defined objectives comply with the NIST standards about brokers. On theother hand, depending on the requirements of the customer, different types of services are provided by cloudproviders, which often need to be combined. Therefore, the use of service composition is increasing as a populartechnology to composite and integrate the distributed and heterogeneous services.The most important advantages of applying the service composition technique are reducing cost and timeas well as improved performance. The composition of cloud services has not taken place in the early days ofthe broker appearance (2009 year). Indeed, since the 2012 year, with the advent of the inter-cloud environments,the complex demands of customers, the expansion of cloud environments and increased competition betweenproviders in providing better services, this topic have become very important. Figure 3 demonstrates that thetrend of publishing studies on composition and integration topics from 2009 to 2019 has been almost ascending.The results of Figure9 and Figure 10 reveal that among the main tasks of the broker (level-one topics extracted ofresearch tree), researchers have paid close attention to providing effective solutions for combining and integratingservices (Composition and integration topic). 133 published studies have been included in the topic of compositionand integration in which the objectives of studies comply with the composition defined by NIST (Figure 13 inRQ6). More precisely, out of 133 studies on composition and integration topic, 63% are related to the role ofarbitration, 28% integration, and less than 10% are related to intermediation. Accordingly, as a guideline forresearchers interested in the field of the broker, it can be said that according to the analysis of the includedstudies, the composition and integration, resource allocation, and service selection are more outstanding andactive topics in the broker field. Also, audiences should note that researchers from 2009 to 2019, among the threemain roles aggregation, arbitration, and intermediation introduced by NIST, have published the most researchstudies concerning intermediation (RQ6). Because intermediation is the simplest and most operational task ofthe broker. Integration and arbitration are new phenomena that have been welcomed and considered by manyresearchers since 2013 (RQ6). , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 25
The results of RQ5 reveals that the majority of studies (71%) have applied simulations to evaluate their methods,and a small number of studies have used testbed (10%) and real-world methods (10%). It should be noted that 10%of the research studies have not mentioned the used evaluation method. Real-world methods usually have morebenefits. Because these methods have a more accurate evaluation and are less exposed to bias and manipulation ofparameters. They can also be a good indicator of the reality of cloud environments. However, due to the dynamicnature of execution time conditions, applying real-world methods to test and evaluate proposed solutions isdifficult and costly which leads to the use of simulation methods such as CloudSim to arrive this goal. On theother hand, the use of simulation methods leads to ignoring the actual conditions at the execution time andcauses a gap between "what has been evaluated" and "what actually exists" .As mentioned earlier, due to the dynamic conditions of the execution time, most researchers have usedsimulation methods to evaluate their work. However, researchers must pay attention to the objectivity of the usedevaluation method. For example, researchers who have used simulation methods to evaluate their solutions shouldprove the objectivity and quality of their proposed solutions to industry experts. Figure 11 illustrates a varietyof techniques including metaheuristic algorithms, similar frameworks and items, semantic and fuzzy methods,etc. to design a cloud broker. Evaluating the mentioned techniques in real-world maybe generate different resultcompared to evaluating them using the simulation methods. Observations in Figure 11 demonstrate that 31%of the studies have used frameworks and similar items to construct a cloud broker, and after that, the use ofmetaheuristic algorithms with 21% of researches have the second rank in meeting broker tasks.
The most common task of a broker is to meet the needs of both cloud providers and customers. To achieve thisobjective, researchers should be aware of the current conditions of cloud environments to construct a perfectcloud broker. To use the techniques introduced in research studies, it is necessary to have special conditions.For example, with the expansion of cloud environments and the increase in customer expectations, often acloud service alone is unable to meet the needs of customers. Hence, the use of techniques introduced in theconstruction of brokers is essential for selecting cloud services and combining them. The analysis of RQ4 revealsthat in 31% of the research studies conducted, the composition and integration topic is considered as one ofthe most important tasks of the broker. Also, 19% of the included studies have concerned the selection of cloudservice through the broker. Considering the analysis of the included studies, the primary responsibilities of thebroker are divided into ten important and primary tasks that each of them has been described in RQ3.Interpretations of RQ5 reveals that a significant portion of the research is focused on to make and applybrokers in large-scale cloud environments. On the other hand, the results of RQ8 show that in large-scale cloudenvironments with high complexity, generally the use of distributed brokers is more common than centralizedbrokers. Since the centralized broker is the most basic type of broker and implementation of it is easy, based onthe results shown in Figure 16, a lot of research has been done in the field of the centralized broker. It should benoted that since the centralized broker itself has all the required information, it has a simple function that has ledto many studies on this type of brokers. However, a distributed broker does not need to have all the informationto make a decision and can do its job independently of other brokers. Such a feature of distributed brokers isinteresting and could be considered more in the future.
Since 2006, with the advent of cloud computing, IaaS services have been the first type of service provided forcustomers on the Internet. Over time, various levels of cloud service have been revealed, and we have dividedthem into five categories in RQ7, including SaaS, PaaS, IaaS (IaaS and CaaS), XaaS (all: anything as a service), , Vol. 1, No. 1, Article . Publication date: February 2021. and Other (special services e.g. NaaS (Network as a Service), DaaS (Desktop as a Service)). The results of theanalysis on the included studies illustrate that most researchers, of the five cloud service categories, have inclinedto provide brokers at the IaaS layer. Since, the services placed in the IaaS layer are more popular or common thanother layers of service. As a result, according to the results obtained from RQ7, Figure 15 demonstrates that 46%of studies have proposed the cloud broker in the IaaS layer. Approximately 20% of the studies have proposed thebroker at the SaaS layer, and the rest of the studies (34%) have examined the broker at other levels.Another important aspect that should be considered is centralization or distribution of brokers in cloudenvironments (RQ8). Generally, the most basic and simplest broker is a third-party that communicate betweencloud customers and service providers. The centralized broker is the simplest broker to be introduced and used incloud environments. We compared previously two types of brokers in detail in Section 4.4 and Section 3.8.
We have carried out a systematic review on cloud broker research to guide researchers, stakeholders and educatorsinterested in this field. Due to the wide range of search spaces under review, different groups of audiences will beable to receive appropriate and worthy implications from the results and discussions presented in Section 3 andSection 4. Each of the results presented in the previous sections can significantly assist different audiences inthis research field. In this section, the implication of our SMS are presented for researchers, stakeholders andeducators. • There are relatively large differences between the rates of studies conducted in different countries (RQ2).Countries such as China (289 studies), the United States (191 studies), and Australia (170 studies) are among themost active countries in the field of research. We have deduced that the advancement of technology in the industryof these countries compared to other countries and the existence of a strong relationship between academicenvironments and industry can cause a high volume of research in these countries. With the advancement oftechnology and industry in a country, conducting academic research to achieve efficient methods and to meet theneeds of the industry is critical. Therefore, more industry progresses in using cloud technologies to meet theneeds of individuals will have a direct impact on the level of acceptance and motivation of researchers in thatcountry to research in the desired field. On the other hand, to offer efficient methods for the cloud broker, theresearchers must employ a suitable platform for evaluating these research methods before entering the industry.Therefore, considering budgets to provide the appropriate infrastructure to evaluate academic research in order toadvance industry goals is a vital step. As RQ5 reveals, since these countries have suitable and rich infrastructuresuch as cloud data centres, can better support and implement research works in the industry. Also, the existenceof research laboratories related to cloud computing such as the laboratory of Buyya has also been influential. • Combining cloud services to meet the complex needs of users is possible in both single-cloud and multi-cloudenvironments. However, considering the extent of inter-cloud environments, environmental conditions at theimplementation time of broker in the inter-cloud environments are more variable and unpredictable than single-cloud environments. Therefore, when a broker is an orchestration, in addition to selecting the service, shouldappropriately combine services and increase the resistance to failure in the broker. • Due to the widespread use of cloud services by cloud customers and startups, as well as the requisiteness tocombine services to provide better services, management of failures during execution of the broker is an urgentnecessity. Because in the process of combining services, in case of failure to run only one service, the process ofrunning the broker will fail. Therefore, as a guideline for researchers and audiences interested in broker field,assiduity to the mechanisms of management and detection of failures during the implementation of the webservices composition is vital. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 27 • The results of RQ1 illustrate that the emergence of brokers has been since 2009 and soon will be one of themain components of cloud computing in negotiation and business to market cloud services. With considerationto commonly utilized solutions of recent decades, stakeholders should use the acquired experiences to improvecloud-based services. • Current research on the development and use of cloud brokers is theoretical and academic. However, applyingthe presented solutions in studies is not commonly observed in the industry. Cloud practitioners and stakeholdersmust play a key role in improving current technology in cloud computing. Therefore, practitioners in the fieldof cloud computing, and especially the broker field, should participate in top conferences in this field (RQ1)and present their perspective and preferences on the current and future research approaches to arrive moreadvancement in the industry. The presence of stakeholders and practitioners in top conferences and workshopshas a great impact on the orientation of the algorithms presented by researchers to adapt to the dynamic and realconditions of cloud environments. Experiences of practitioners can significantly affect the method of constructingbrokers and the method of classifying them (RQ2). On the other hand, practitioners can acquire more profit bycollaborating with researchers in academia. • Presently, most solutions offered in broker research have adequate quality for use in real-world environments(RQ1, RQ6). It should be noted that there is a lack of empirical evidence from the industry. Industry experts(practitioners) and stakeholders should collaborate with researchers to increase insights of researchers intoindustry-friendly metrics.
With over 10 years of experience, cloud brokers are becoming one of the most promising solutions for tradingin complex cloud environments (RQ1). These constructed brokers have been the result of research by the mostsuccessful researchers and pioneers in the field of cloud brokers. Accordingly, an advantage of using academicexperience in constructing an industry-friendly broker is the transfer of knowledge and experience to novicedevelopers. As a guideline, the results of our SMS suggest that educators and teachers in courses such ascloud computing and distributed systems should include cloud brokers as an important component of cloudenvironments in their curriculum. Wide research conducted in the broker field can employ as a training resourcefor teachers and educators who are teaching cloud computing course. Considering the acquired results of theanalysis of the included studies in Section 4 and Section 5, the teachers and educators can inform students ofa variety of unpredictable conditions and the occurrence of failures and faults that may occur during brokerimplementation.
We conducted a systematic mapping study on the field of cloud brokers that are placed in the range of 2009 tothe end of 2019. We extracted a total of 1,298 related studies from search spaces and then selected 496 studiesbased on the quality criteria set out in our search strategy. An important part of our SMS is presenting a powerfulresearch methodology. Our SMS contains a comprehensive three-layer including manual search strategy, backwardsnowballing, and database search of reputable scientific libraries. We evaluated our search methodology and theresults showed that our search methodology can find more than 80% of the studies conducted in the broker field.We have also provided a comprehensive supplementary document containing complete details of the informationextracted and reviewed in each phase of our systematic review.We have determined a set of eight research questions and responded to them during our SMS. The first threeresearch questions (RQ1, RQ2, and RQ3) include reviewing the amount of research conducted in the brokerfrom 2009 to 2019, extracting the most important topics and tasks of the cloud broker and introducing the , Vol. 1, No. 1, Article . Publication date: February 2021. most important researchers and pioneers in the cloud broker field. Also, by answering RQ1, RQ2, and RQ3,broker audiences will acquaint with countries that are active in the broker field. Two other questions (RQ4, RQ5)investigate the amount of research conducted in each of the important topics of brokers, the rate of growth ofresearch in each topic over time, the types of evaluation methods used in research studies and finally the appliedtechniques to construct cloud brokers. We also compared our SMS with several new and valid related reviewsand examined the depth of the methodology used, the number of studies investigated, and the various aspectsconsidered in each conducted review. In the last three questions (RQ6, RQ7, and RQ8), we examined and analyzedthe tasks and topics extracted from the included studies from the perspective of the definitions of NIST. In RQ7,we identified the different layers of service for the cloud broker and determined the number in included studiesconsidering each layer. In RQ8, we looked at two important aspects of cloud brokers, namely centralized anddistributed, and analyzed broker studies according to two defined aspects.Our SMS shows that the field of cloud brokers is active and growing in various geographical locations, andthe development of cloud brokers needs to be done under the latest research achievements. Researches areincreasingly using brokers to develop interactions between customers and cloud service providers. Systematicmapping studies, such as our SMS, can be used as a basis for a more specific review of systematic literature. Infuture work, each of the top-level topics extracted from our research tree can be further explored to answer morespecific research questions.
REFERENCES [1] Er Kamna Agarwal and Sugandha Singh. [n. d.]. A Survey on Infrastructure Platform Issues in Cloud Computing. ([n. d.]).[2] J Akilandeswari and C Sushanth. 2014. A review of literature on cloud brokerage services.
International Journal of Computer Science andBusiness Informatics
10, 1 (2014), 25–40.[3] Bandar Aldawsari, Thar Baker, and David England. 2015. Towards a holistic multi-cloud brokerage system: Taxonomy, survey, and futuredirections. In . IEEE, 1467–1472.[4] Gaetano F Anastasi, Emanuele Carlini, Massimo Coppola, and Patrizio Dazzi. 2017. QoS-aware genetic cloud brokering.
Future GenerationComputer Systems
75 (2017), 1–13.[5] Fariba Aznoli and Nima Jafari Navimipour. 2017. Cloud services recommendation: Reviewing the recent advances and suggesting thefuture research directions.
Journal of Network and Computer Applications
77 (2017), 73–86.[6] Adam Barker, Blesson Varghese, and Long Thai. 2015. Cloud services brokerage: A survey and research roadmap. In . IEEE, 1029–1032.[7] Adrija Bhattacharya and Sankhayan Choudhury. 2017. Service Provisioning in Cloud: A Systematic Survey. In
Advanced Computing andSystems for Security . Springer, 37–63.[8] Sameer Singh Chauhan, Emmanuel S Pilli, Ramesh Chandra Joshi, Girdhari Singh, and Mahesh Chandra Govil. 2019. Brokering ininterconnected cloud computing environments: A survey.
J. Parallel and Distrib. Comput.
133 (2019), 193–209.[9] Tore Dyba, Barbara A Kitchenham, and Magne Jorgensen. 2005. Evidence-based software engineering for practitioners.
IEEE software
22, 1 (2005), 58–65.[10] Mona Eisa, Muhammad Younas, Kashinath Basu, and Hong Zhu. 2016. Trends and directions in cloud service selection. In . IEEE, 423–432.[11] Rania Fahim El-Gazzar. 2014. A literature review on cloud computing adoption issues in enterprises. In
International Working Conferenceon Transfer and Diffusion of IT . Springer, 214–242.[12] Abdessalam Elhabbash, Faiza Samreen, James Hadley, and Yehia Elkhatib. 2019. Cloud brokerage: A systematic survey.
ACM ComputingSurveys (CSUR)
51, 6 (2019), 1–28.[13] Yehia Elkhatib. 2016. Mapping cross-cloud systems: Challenges and opportunities. In { USENIX } Workshop on Hot Topics in CloudComputing (HotCloud 16) .[14] Frank Fowley, Claus Pahl, Pooyan Jamshidi, Daren Fang, and Xiaodong Liu. 2016. A classification and comparison framework for cloudservice brokerage architectures.
IEEE Transactions on Cloud Computing
6, 2 (2016), 358–371.[15] Frank Fowley, Claus Pahl, and Li Zhang. 2013. A comparison framework and review of service brokerage solutions for cloud architectures.In
International Conference on Service-Oriented Computing . Springer, 137–149.[16] DV Geetha, RM Hayat, and M Thamizharasan. 2014. A survey on needs and issues of cloud broker for cloud environment.
Int. J. Dev.Res
4, 5 (2014), 1035–1040., Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 29 [17] Nadir Ghrada, Mohamed Faten Zhani, and Yehia Elkhatib. 2018. Price and performance of cloud-hosted virtual network functions:Analysis and future challenges. In . IEEE, 482–487.[18] Nikolay Grozev and Rajkumar Buyya. 2014. Inter-Cloud architectures and application brokering: taxonomy and survey.
Software:Practice and Experience
44, 3 (2014), 369–390.[19] Thomas Hofmann. 2001. Unsupervised learning by probabilistic latent semantic analysis.
Machine learning
42, 1-2 (2001), 177–196.[20] Abbas Javan Jafari and Abbas Rasoolzadegan. 2020. Security patterns: A systematic mapping study.
Journal of Computer Languages
InternationalJournal of Computer Applications
975 (2017), 8887.[22] Anthony D JoSEP, RAnDy KAtz, AnDy KonWinSKi, LEE Gunho, DAViD PAttERSon, and ARiEL RABKin. 2010. A view of cloudcomputing.
Commun. ACM
53, 4 (2010), 50–58.[23] Amin Jula, Elankovan Sundararajan, and Zalinda Othman. 2014. Cloud computing service composition: A systematic literature review.
Expert systems with applications
41, 8 (2014), 3809–3824.[24] Mrs Akanksha Kherdikar. [n. d.]. A Comparative Study of Traditional Cloud Service Providers and Cloud Service Brokers. ([n. d.]).[25] Barbara Kitchenham, O Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman. 2009. Systematic literaturereviews in software engineering–a systematic literature review.
Information and software technology
51, 1 (2009), 7–15.[26] Barbara A Kitchenham, David Budgen, and O Pearl Brereton. 2011. Using mapping studies as the basis for further research–a participant-observer case study.
Information and Software Technology
53, 6 (2011), 638–651.[27] Anupriya Koneru and M Sreelatha. 2017. A comprehensive study on cloud service brokering architecture. In . IEEE, 47–53.[28] Fang Liu, Jin Tong, Jian Mao, Robert Bohn, John Messina, Lee Badger, and Dawn Leaf. 2011. NIST cloud computing reference architecture.
NIST special publication
Journal of Systems and Software
125 (2017), 93–118.[30] Luís Monteiro and André Vasconcelos. 2013. Survey on important cloud service provider attributes using the SMI framework.
ProcediaTechnology . IEEE,25–31.[32] Victoria Paulsson, Vincent Emeakaroha, John Morrison, and Theo Lynn. 2016. Cloud Service Brokerage: A systematic literature reviewusing a software development lifecycle. (2016).[33] Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in softwareengineering: An update.
Information and Software Technology
64 (2015), 1–18.[34] Nagarajan Rajganesh and Thirunavukarasu Ramkumar. 2016. A review on broker based cloud service model.
Journal of computing andinformation technology
24, 3 (2016), 283–292.[35] R Rakhi, Litty Mariam Biju, Sreelekshmi S Kumar Rinta, Mariam Biju, and Minu Lalitha Madhavu. 2016. A Survey on Various CloudAspects.
International Journal of Advanced Research in Computer Science
7, 5 (2016).[36] Ali Ahmadian Ramaki, Abbas Rasoolzadegan, and Abbas Ghaemi Bafghi. 2018. A systematic mapping study on intrusion alert analysisin intrusion detection systems.
ACM Computing Surveys (CSUR)
51, 3 (2018), 1–41.[37] Bahman Rashidi, Mohsen Sharifi, and Talieh Jafari. 2013. A Survey on Interoperability in the Cloud Computing Environments.
International Journal of Modern Education & Computer Science
5, 6 (2013).[38] Adel Nadjaran Toosi, Rodrigo N Calheiros, and Rajkumar Buyya. 2014. Interconnected cloud computing environments: Challenges,taxonomy, and survey.
ACM Computing Surveys (CSUR)
47, 1 (2014), 1–47.[39] Cheng Zhang and David Budgen. 2011. What do we know about the effectiveness of software design patterns?
IEEE Transactions onSoftware Engineering
38, 5 (2011), 1213–1231.[40] Jiangtao Zhang, Hejiao Huang, and Xuan Wang. 2016. Resource provision algorithms in cloud computing: A survey.
Journal of Networkand Computer Applications
64 (2016), 23–42.
APPENDIX A
Already, distinct works have been done for conducting SMS and designing a unique research methodology [14, 34].One of the most prominent of these research methodologies is related to Peterson et al. [14]. The researchmethodology used in this paper is adapted from three SMS that were done by Peterson et al. [14], Ramaki etal. [36], and Javan et al. [20] and some updates and improvements in some phases have been done. Generally, the , Vol. 1, No. 1, Article . Publication date: February 2021. suggested SMS consists of two main steps i.e. planning step and conducting step. Each of these steps is completelyexplained in the rest of this section.
As can be seen in Figure 17, the presented SMS consists of different steps. At the first step, the scope and researchquestions are specified and then three steps, namely planning the mapping study, evaluating the mapping study,and conducting the mapping study steps are simultaneously executed. Each step has a specified strategy toperform the mapping study that in the following, has been described in detail.
Defining some research questions in a review research work is essential. These research questions should beresponded during the review by the researchers. Indeed, the objective of defining RQs is to determine the researchgoals. Therefore, one of the most important steps in an SMS process is designing some useful questions so thatafter answering them, all challenges, issues and important topics in the desired field be determined. In this SMS,8 comprehensive and different RQs have been defined, which responding them can cover all our objectives. Table1 in article describes these RQs and explains the rationality of each of them.
To conduct the search process, specifying the exact search strategy is vital which should be carried out duringthe process of review. In this paper, as previously stated, three distinct search strategies (i.e. database search,backward snowballing search, and manual search) have been designed to find all related studies. It’s worthmentioning, considering the investigation of research studies, the advent of the cloud broker is 2009. Therefore,this SMS covers all published research articles from 2009 until end of 2019 and a complete review have beenconducted on the cloud broker research works during this period. At the first step, all related papers from 2009 to2019 are retrieved by combining the results of backward snowballing and manual searches. Moreover, a distinctsearch on well-known databases like Google Scholar, Springer Link, IEEEXplore, ACM Library, and ScienceDirectis done to find unseen related articles and to identify the possible shortcoming in the snowballing and manualsearch strategies.Figure 18 illustrates the general strategy which comprises seven steps. It should be noted that the determiningrelated search spaces (journals, conferences, and workshops) are not necessarily carried out after the manualsearch step. Hence, an S-Flag is defined and set with zero in step 2 and one in step 5. Current search strategycomprises seven primary steps that are acquired by investigating the base reviews studies [14, 20, 36] and applyingsome improvements to them. • In this step, a set of initial secondary studies (see Table 1 in article) are randomly selected and investigatedto determine a set of initial keywords that are used in the next steps for finding the related studies. Also, aphase variable is used to show the number of passed steps. • In this step, for finding the related studies with high quality, two types of quality measures are specifiedthat are called qualification and relation criteria. As mentioned previously, the search strategy should beable to extract and find all related search spaces (journals, conferences, and workshops) to retrieve allstudies related to the broker topic. Therefore, to arrive at the high-quality studies, the existence of qualitymeasures is necessary that filters journals, conferences, and workshops namely, search space, in termsof quality factors defined in the SMS. To reach a set of initial search spaces, all cited studies in the initialsecondary studies are extracted. After reviewing all initial secondary studies, if a new search space is found,then step 3 should be done, else the search process will terminate and step 5 should be run to conductthe backward snowballing on the included studies. Consequently, a flag called S-Flag has been defined to , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 31
Planning the Search ProcessPlanning the
Study Selection
Process
Planning the
Data Extraction and ClassificationSpecify Scope and
Research QuestionsEvaluate the Mapping Study
Specify Data Extraction Forms
Specify Search Study Selection Evaluation Strategy
Specify Study Selection ProcessSpecify Search String
Specify Search Space
Specify Search Space
Inclusion/Exclusion CriteriaSpecify Search Strategy
Specify Study
Inclusion/Exclusion Criteria
Specify Data Classification Type and StrategySpecify Data Extraction StrategyQuality Acceptable?Answer and Discuss the secondary research QuestionsAnswer and Discuss the Primary QuestionsApply the Analysis and Classification ProcessExtract Data from Each Study Evaluate the Search and Study Selection
Conduct Search and Study
Selection
Evaluation passed?
First Evaluate?Planning Mapping Study NO YesNO
Yes NO Yes
Conducting the Mapping Study
Fig. 17. Research methodology [20] , Vol. 1, No. 1, Article . Publication date: February 2021. prevent ineffective searches during the search process. If the search process is in the backward snowballingstep, the amount of S-Flag is equal to one, else it is set to zero. S-Flag in this step will be initialized withzero. When a backward snowballing is done on the included papers, the S-Flag will be set to one. • In this step, the extracted search spaces should be evaluated considering two defined quality measures andthe compatible search spaces should be selected. The set of search spaces is updated by adding the extractednew search spaces. The output of this step is a set of high-quality related search spaces that concludes allrelated papers in the field of cloud broker. The manual search is done on the selected search spaces by theinitial set of keywords corresponding queries defined in Table 2 in article to find the related papers. • In this step, the study selection process is done. If a study satisfies all quality criteria, therefore, it is anincluded study and the search process jumps to step 5. Moreover, if there are some new keywords in theincluded studies, then they are extracted and added to the keywords set. This action concludes a completeset of related keywords in the field of cloud broker that is led to a comprehensive search study set. If thepaper has been acquired of the backward snowballing, the process jumps to step 2. • In this step, backward snowballing is done on the all new found included papers and S-Flag is set to one.If the backward snowballing results new papers therefore, the execution jumps to step 4 to conduct thesearch space process, otherwise step 6 is done. • with applying a set of pre-defined keywords, a database search is done and new unseen papers are extracted. • at the last step, the study selection process should be executed on the recently found papers and after that,the search process ends.As previously mentioned, an auxiliary document called SuppFile has been provided and throughout the paper,may be referred to it. For conducting the database search, different databases for finding the related papers inthe desired field have been investigated such as ScienceDirect, ACM Digital Library, IEEE Xplore, Springer Link,and so on. A complete list of extracted studies can be seen in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 to 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 (Supplementarydocument in SuppFile folder). As previously mentioned in the description of the search strategy, some qualitycriteria for evaluation of search spaces quality and found papers have been applied. Therefore, the database searchis done just on the high-quality search spaces that have been more related in terms of scopes. The
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 to 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 shows a complete list of all extracted search spaces in our research field. At the beginning of conducting the SMS process, the search space set is empty. Therefore, after selecting somesecondary studies some journals, conferences and workshops are added to the search space list by investigatingarticles cited in the references section of secondary studies.
The objective of this SMS is to review all papers in the field of cloud broker. To search related papers in themanual and database search phases, a set of keywords have been defined that can be seen in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 . Thesekeywords are gradually completed during the progress of the review process. The keyword set is completed bythe three mentioned phases. Also, as previously mentioned, some queries have been defined to search relatedstudies in the database of different publishers. These queries are shown in Table 2 in article and are a combinationof elementary defined keywords in the first step of finding papers. As shown in Figure 17, this phase comprises three primary steps namely, specifying the search space inclu-sion/exclusion criteria , specifying study inclusion/exclusion criteria , and specifying the study selection process . In thefollowing, each of the phases will be explained in detail. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 33 Specify the Initial Set of Papers ad
Keywords Set; Phase=0
Extract the Search Spaces; S-Flag=0; Backward Snowballing on New Papers;
S-Flag=1;
Phase++ Apply the Study Selection ProcessAdd Them to Search Spaces and Conduct
Manual Search using KeywordsAny New Search?
S-Flag==1Any New Paper?
Apply the Study selection Process
Conduct Database Search and Identify
New Papers
Any New Paper Found?
YESYES
YES NO NONO YES NO
Fig. 18. The Search Strategy [20] • Specifying the search space inclusion/exclusion criteria.
For beginning the review process, a set of review and survey papers in the desired field are found and someof them are selected by experts. These selected review and survey papers that have been placed in Table 1 inarticle are called initial secondary studies. In the first step, all search spaces in the reference list of the secondarystudies are investigated and extracted into a list of new search spaces. Then some search spaces with good qualityand related scope are selected among them. To arrive at this goal, some quality criteria for determining theinclusion/exclusion state of search spaces have been specified. The exclusion criteria are illustrated in Table 12 forjournals and Table 13 for conferences and workshops. The result of applying the exclusion criteria is removingall unrelated or low-quality search spaces. , Vol. 1, No. 1, Article . Publication date: February 2021.
Table 12. Journal Exclusion Criteria (JEC)
If the journal is not indexed in the JCR If the scope of the journal is not related to our desired field
Table 13. Other Exclusion Criteria (OEC) ((Qualis < A5) OR (ERA < A) OR [(H5_Index <
15) AND ((Qualis < A5) OR (ERA < A))] OR (Metrics Not Available)) (Aims and scopes are not related)To evaluate the quality of journals, the JCR metric has been used and ERA and H5-Index metrics are used forconferences and workshops (these are shown in Table 12 and Table 13). It should be noted that there are somevaluable points that are explained below: • The field of cloud broker comprises a large pool of research studies, therefore some thresholds for theexclusion criteria considering two principals have been empirically selected: 1) a small change in theexclusion thresholds should not have a big effect on the number of excluded/included papers and 2)applying these thresholds should not exclude a lot of highly cited papers. • Although the desired field in this SMS is cloud broker, but search spaces with the scope and aim of webservices or distributed and parallel computing have been considered. There are also search spaces in otherresearch areas which are included, because they have published some papers in the field of cloud computing.An example is IEEE Transaction on Smart Grid. • There are some search spaces such as the International Conference on Software Engineering (ICSE) that havescarcely published papers in the field of cloud computing. Because its scope is about software engineering.Therefore, such search spaces are excluded. • In rare cases, we found a search space with two names. For example, the ACM International Symposium onHigh-Performance Distributed Computing and International Symposium on High-Performance Parallel andDistributed Computing are two names for the search space of HPDC. Also, there is a search space that itsname had been changed. The initial name for GLOBECOM search spaces is IEEE Global TelecommunicationsConference, and after 1972, its name has been changed to Global Communications Conference. As anotherexample, after 2007, two search spaces namely, European Conference on Machine Learning (ECML) andEuropean Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD) have beenmerged titled European Conference on Principles and Practice of Knowledge Discovery in Databases(PKDD). • Specifying the study inclusion/exclusion criteria
After finding the related and included search spaces, they are searched using constructed queries in Table 2 inarticle to find the related papers.
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 presents complete information of all included papers. The inclusioncriterion for an extracted study is its relevance to our designated scope. To determine the inclusion/exclusionstate of each extracted paper, title, abstract, and keywords have been generally investigated. In some cases, theentire paper has been reviewed to ensure its inclusion or exclusion status. A complete list of all excluded paperscan be seen in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 . It is worth mentioning that all extracted review studies during conducting thesearch process in this SMS have been excluded. A complete list of all extracted review studies in the desired fieldcan be seen in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 . Table 14 illustrates the exclusion criteria for extracted studies. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 35 Table 14. Exclusion criteria for extracted studies
The study is not a primary study (e.g. survey) Study cannot be accessed (e.g. not indexed) The study is out of our primary scope( e.g. security) The study belongs to an excluded search space The contribution of the study is not related to the cloud broker.By applying the exclusion criteria for search spaces and determining the included search spaces, the manualsearch for finding related studies are just conducted for the included search spaces. However, by using thebackward snowballing search and database search some studies may be retrieved from the excluded search spaces.Therefore, for checking this situation, an exclusion criterion is defined (see Table 14, row 4). According to thefirst criterion, secondary studies (for example reviews, surveys) have not been investigated in this SMS. • Specifying the Study Selection Process
Figure 19 illustrates the study selection strategy that comprises two main steps. In the first step, the extractedstudies are reviewed by reading their titles, abstracts, keywords and in some cases full-text. If the paper is irrelevantor its relevancy is uncertain, therefore the judgment of a third-party is needed to determine its relevancy. Inthe second step of the study selection strategy, determining the exclusion of paper is done by using the definedcriteria in Table 14.
In this review, two prominent and most valid metrics are used to evaluate the search strategy. In other words,both objective evaluations (i.e. quantitative criteria) and subjective evaluation (managed by expert review) aredone using Eq. 3 and Eq. 4 respectively. To have a more objective evaluation, the quasi-sensitivity metric is usedto evaluate the applied search and study selection.
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = The Number of Studies in Our SMSThe Number of Studies Overall ×
100 (3)
𝑄𝐺𝑆 = number of discovered papers in the search phasenumber of discovered papres in the evaluation phase ×
100 (4)QGS alludes to a set of studies published in the well-known research communities. To create this gold standard,the home pages of active researchers in the area of cloud computing have been visited and their papers in the cloudbroker field have been extracted. After extracting studies from home pages and applying the inclusion/exclusioncriteria, the remaining unseen papers comprise our GCS according to Eq. 4. The aim of applying the QGS iscalculating the quasi-sensitivity and then comparing the obtained result with a predefined threshold. Accordingly,if the result falls below the threshold, the search and study selection process should be repeated using QGS. Byfollowing [5], an acceptable threshold should be between 70% and 80%. In the evaluation phase, 32 articles werefound, of which 26 articles were found in the main search phases of the systematic review process, therefore, weachieved a sensitivity of 81.25% which is above our predefined threshold. It means that the probability of notfinding a paper related to the cloud broker is less than 20%. Therefore, it can be concluded that the acquiredresults from this review have enough accuracy and validity. , Vol. 1, No. 1, Article . Publication date: February 2021.
Evaluate Relevance by Title,
Keywords, Abstract Evaluate Quality by
CriteriaMark as Excluded
Include PaperDiscard Paper Uncertain
Mark as Included
Evaluate Relevance by Full-
Text [Include][Exclude] [Include][Exclude] [Exclude][Include]
Fig. 19. The study selection strategy [20]
After conducting the study search process and determining the included papers, to answer the RQs, someinformation is needed that should be extracted from these papers. Table 8 in article illustrates that whichinformation is useful for answering which RQs. It should be noted that the "Topic" word in Table 8 in articlerepresents an acquired concept from the "keywords" field using a keywords clustering algorithm that is acommon approach for topic-dependent classification [14].
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ( , 𝑡𝑜 ) include comprehensively all neededinformation to answering RQs. The evaluation criteria that have applied to evaluate this SMS are similar to [14]. Details of this information canbe seen in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 , ( 𝑇 𝑡𝑜𝑇 ) . During the review process, if the quality level is inadequate, the search processmust be revised to solve existing problems. As can be seen, the applied strategies for this SMS have been described in Sections 7 and 8 In the following, theobtained results of applying these strategies have been shown. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 37
Table 15. Sample Extracted Journal Search Spaces and Results of the Search Space Selection Process
𝐼 𝐹
𝐽𝐶𝑅 − 𝑄 𝐽𝐶𝑅 − Topic Publisher Depth ReaseonofExclusion6
Future GenerationComputer Systems 1872-7115 4.639 Q1 Computer sciencea Theory andmethods Elsevier 0 - IEEE InternetComputing 1941-0131 1.929 Q1 Computer science aSoftware engineer-ing IEEE 0 - Annals of Telecom-munications 1958-9395 1.168 Q4 Telecommunication Springer 0 JEC2 IEEE InternetComputing N/A - - N/A Springer 0 JEC1
Table 16. Sample Extracted other Search Spaces and Results of the Search Space Selection Process Name H5_Index Qualis ERA Indexing Depth ReaseonofExclusion46
ACM SIGKDD Conference on Knowledge Dis-covery and Data Mining (KDD) 77 A1 A ACM 0 - International Conference on High Perfor-mance Computing and Communications(HPCC) 20 - B IEEE 0 OEC1 International Conference on Information andKnowledge Management (CIKM) 49 - A ACM 0 OEC2 International Conference on Data Engineering(ICDE) 53 A1 A IEEE 0 - Conference, Workshop and Symposium
The first step when running the search process is providing an initial set of papers that are acquired from aset of secondary studies that have been found using an informal search process by using the keywords cloudbroker along with keywords such as survey and review . A depth variable describes the type of search in our searchmethodology. After acquiring a set of secondary studies (shown in Table 1 in article), the initial value of depth isset to zero and related cited papers from those studies are extracted that constitute an initial set of included studies(primary studies). After that, the backward snowballing and manual search is done on the included papers in theinitial set. After finishing manual and snowballing search, for complementing the results set, a database search isdone. Table 15 and Table 16 demonstrate sample records of journals, conferences and workshops respectivelythat their inclusion/exclusion state have been determined based on the exclusion criteria defined in Table 12 andTable 13. The
Reason of Exclusion column of these tables determines the reason for exclusion for each searchspace. , Vol. 1, No. 1, Article . Publication date: February 2021.
Table 17. Sample extraction table for journal papers
Paper Title Journal Name Year ExclusionCriteria
Privacy-preserving and sparsity-aware location-based prediction method for collaborative recom-mender systems Future Generation Computer Systems 2019 PEC1A trust centric optimal service ranking approach forcloud service selection Future Generation Computer Systems 2018 -A privacy-preserving cryptosystem for IoT E-healthcare Information Sciences 2019 PEC1Implementation of a real-time network traffic moni-toring service with network functions virtualization Future Generation Computer Systems 2019 PEC1A hybrid multi criteria decision method for cloudservice selection from Smart data Future Generation Computer Systems 2019 -For example, in Table 15, the journal with JID = 11 is excluded since its scope is not related to the defined scopeof the cloud broker. As another example, the journal with JID =12 is excluded, because it has not been indexed inJCR. Table 16 illustrates that a conference with OID = 52 is excluded because its metrics i.e. Qualis and ERA areless than a defined amount in Table 13. Also, OID=62 is excluded because its scope is not related to the scopeof the cloud broker field (refer to
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 ). Complete information about all extracted search spaces andreason for exclusion and inclusion of journals, and other (conferences, and workshops) have been provided in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 respectively. Table 17 and Table 18 demonstrate a sample of extracted papersbased on the described search spaces in Table 14 and the exclusion and inclusion states for each paper consideringthe exclusion criteria has been determined. For example, in Table 17, the presented contribution by the first studyis not related to the aim and scope of a cloud broker, so this study is marked as an excluded paper. 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 present list of extracted papers and comprehensive information about thereason for the exclusion of each study. The main aims and scope of the cloud broker field in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 have been determined. Moreover, the aims and scope of each of the search spaces have been introduced in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 and the reason for the exclusion of each search space has been determined inthem.After constructing Table 17 and Table 18 and determining the included studies, by investigating the keywordsof the included papers, some new keywords are extracted and added to the set of keywords for later use. 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 and 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 , ( 𝑇 𝑡𝑜𝑇 ) illustrate complete information of all other papers (conferences andworkshops) and journal papers respectively. 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 presents a complete list of all included papers (journals,conferences, and workshops). The obtained results of the evaluation phase ( 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 , ( 𝑇 𝑡𝑜𝑇 )) have been used to extract the active researchers.A list of active researchers can be seen in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 and the author’s country can be found in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 .The validation of this SMS has been evaluated by conducting the evaluation phase. In this phase, the home-page of authors manually have been probed for finding unseen papers during the execution of three searchphases. 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 illustrates a list of unseen papers during conducting search phases. Based on the providedinformation, there are 6 included studies that our SMS failed to find them. In other words, during the reviewprocess, their search space has not been taken into consideration. Again, the search strategy was done on these , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 39 Table 18. Sample extraction table for Conference papers
Paper Title Conference Name Year ExclusionCriteria
A model for evaluating the economics of cloudfederation International Conference on Cloud Net-working (CloudNet) 2015 -Cloudlet Scheduling with Particle Swarm Opti-mization International Conference on Communica-tion Systems and Network Technologies(CSNT) 2015 PEC1Incentivizing Microservices for Online ResourceSharing in Edge Clouds International Conference on DistributedComputing Systems (ICDCS) 2019 -The Elasticity and Plasticity in Semi-Containerized Co-locating Cloud Workload:A View from Alibaba Trace ACM Symposium on Cloud Computing 2019 PEC1unexplored studies that led to some new included search spaces (4 journals, 18 conferences and 7 workshops) tothe search spaces set. The investigation of these new search spaces added 17 new included studies to the studiesset. Considering these conducted processes to evaluate the completeness of the search strategy, we conclude thatthe set of studies has satisfactorily covered the cloud broker field.
After completing the set of studies, the process of data extraction and classification is performed on them.
After organizing data into tables, to respond to the RQs of our SMS, these data should accurately be investigatedand analyzed. The most important objective in the presented SMS is determining the primary topics and sub-topicsin the cloud broker field. In the rest of this section, we explain the process of topic and sub-topic detection.So far, some techniques have been presented to identify topics in the research fields. Detail informationabout this phase is placed in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝑊 ,𝑇 𝑎𝑛𝑑𝑇 ) .Among all proposed approaches, an outstanding approachthat has been proposed by [21, 38] applies a statistical similarity factor to classify keywords and discovers thecharacterizations of topics. The main disadvantage in this approach is the determination of topics and sub-topicsjust according to a small piece of the paper. For example, they just consider a paragraph of the paper to determinethe topic. In this review to determine the topic of a paper, we usually investigate the whole paper.Usually, the similarity-based keywords clustering techniques do not have a high classification power [ ? ],therefore, other techniques are proposed by some researchers [ ? ? ? ] that can group the keyword set into someclusters based on co-occurrences matrix of keywords, and find the related topics and sub-topics considering theirsimilar cognitive orientation [ ? ]. Besides, in each step, the number of clusters corresponds to the determinedissues in the SMS. In these approaches, detection of topics is heuristically done, but in this SMS, we propose anew approach based on co-occurrences matrix and use the knowledge of expert/experts for finding topics andsubtopics. Previously, we have used this technique in our review works [ ? ? ].The last step in the search process is to identify a set of keywords from included papers that may be donedirectly or indirectly. When keywords set are available in an included paper, these keywords are chosen, otherwise,the extraction of keywords must be done manually by an expert. Figure 20 demonstrates the construction processof the research tree. A research tree is a multilevel tree that contains the main topics and sub-topics in the desiredfield. The most important level in this tree is its root. Because all analysis to respond to RQs is done based on the , Vol. 1, No. 1, Article . Publication date: February 2021. Construct Research Tree level by Level Level = level + 1
Detect Next Level Topic for
Each Study based on
Selected MeasureSelect a New Candidate
Measure for Next Level
Topic Detection
Calculate the Percentage of Research Works for Each Topic Exists in Current LevelIdentify Level one Topic of it based on Title, Abstract, Author
Keywords (Full Text if nedded)Select a Study from the Studies set
Consider the Final Included
Studies Set
Is There Any Topic with More than 15%? Are the Studies Almost Evenly Distributed? Is There Any Next Level?
YESNO
YES NO YES NO Activity Decision
Work Flow
Fig. 20. The construction process of the research tree [36] located topics and sub-topics into the root level. The extraction process of level-one topics in research tree can beseen in Figure 21.In this figure, first all keywords of included papers should be extracted and then the most frequent keywordswhich are appeared more than a threshold value are chosen. Then, the co-occurrence matrix should be constructedfrom the frequent keywords. A sample of the co-occurrence matrix is shown in Table 19. 𝑥𝑖 𝑗 is a cell in the 𝑖 throw and the 𝑗 th column and illustrates the number of times that the word of the 𝑖 th row appears with the 𝑗 thcolumn word at the same paper. For example, the service composition keyword appears in 88 studies, whichco-occurs with QoS keyword in 31 studies. A complete co-occurrence matrix that comprises 40 keywords isplaced in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 . After Construction of co-occurrence matrix to detect the cognitive orientation of twokeywords in relation to all other keywords, it should be normalized and the similarity of each keyword pair onthe co-occurrence matrix should be computed. In other words, each keyword has a vector that each element ofit depicts co-occurrences with another keyword. The similarity is computed based on the cosine index, whichis demonstrated in Eq. 5 [ ? ]. The resulting matrix should be classified. We use the K-means algorithm [ ? ] asa common clustering method where K is initialized by an expert and illustrates the number of clusters to beconstructed. , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 41 Extract All Keywords From the Final Included Studies
Extract Top Frequent KeywordsSelect Appropriate Cluster Name as level One Topic NameCalculate Co-Occurrence Matrix for Frequent Keywords
Keywords Clustering
Fig. 21. The extraction process of level-one topics in research tree [36]Table 19. A sample of the co-occurrence matrix
Keywords ServiceComposition QoS ServiceSelection Resourcemanagement SchedulingService Composition
88 31 12 3 0
QoS
31 68 11 7 4
Service Selection
12 11 56 1 1
Resource management
Scheduling
𝑆𝑖𝑚 ( 𝑥, 𝑦 ) = (cid:205) 𝑖 ( 𝑥 𝑖 𝑦 𝑗 ) √︃ (( (cid:205) 𝑖 𝑥 𝑖 )( (cid:205) 𝑖 𝑦 𝑖 )) (5)The number of included papers is 496 that can be seen in 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 . In the following, we explain the processof research tree construction and top-level topic detection considering frequency of keywords. , Vol. 1, No. 1, Article . Publication date: February 2021. Table 20. Thematic similarities between the keywords of included papers
Topic Similar Concept
Service composition and integration Composition + Orchestration + IntegrationService allocation Scheduling + ProvisioningEnergy management Energy management + Green computingSLA management SLA violation + SLA management and etc.Resource allocation Resource allocation + Resource management + VM schedulingPricing Pricing + Cloud marketRecommendation Recommendation + Data management • the keywords of all the included papers (496 papers) are collected. If the paper does not have the author’skeywords, the keywords are produced by an expert. In the keyword’s column in the 𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 , ifa keyword has been manually extracted by an expert, the cell color is yellow. To determine keywordsmanually by an expert, the title, abstract, and context of the paper and also the defined queries in Table 2have been used. The keywords of included papers have been collected in a keyword pool and then thefrequency of each keyword has been computed. For example, the broker keyword has been seen in 100included papers. The size of the keyword pool after the first phase was 1291. • we have categorized keywords that are lexically similar. For example, SaaS, SaaS applications, and softwareas a service are located in the SaaS group. After grouping all keywords, the number of keywords decreasedto 735 items. • Some of the most repetitive keywords that do not give rise to a new concept in the research tree have beenremoved from the keyword list. Example of these keywords are broker, cloud computing, edge computing,distributed system, IoT, mobile cloud computing, and autonomic computing. • The co-occurrence matrix is obtained for words with a repetition threshold greater than or equal to 9. Therepetition threshold is obtained by trial and error, and at threshold 9 the best semantics is coincidentallyfound in the matrix output. A total of 35 keywords with a repetition greater than or equal to 9 formedthe primary matrix. Then, in each iteration, the two words that have the most coincidence compared toother pairs of words, are selected and form a new subject. This process continues until there is no newco-occurrence in the matrix members. After 18 repetitions of the process of finding the highest number ofcoincidences, the final number of 18 items has been obtained. Table 21 shows the compounds obtained as aresult of the co-occurrence matrix and the topics obtained as the level-one topics of the research tree. Byexamining the related topics by experts and aggregating them, finally 10 main topics (the second columnin Table 21) have been created in the cloud broker field that is introduced as the level-one topics of theresearch tree. Topics are divided into two sections: client-centric and provider-centric.Client-Centric topics are activities that the broker performs as a result of the user request. For example, as a resultof a service request by a user, the broker may perform service discovery, service selection, service composition,and so on. Provider-Centric topics are activities that the broker performs as a result of the provider’s request. Forexample, pricing, resource allocation, energy management, etc. It should be noted that service allocation includesservice provisioning and scheduling on the client-side. Figure 1 in article shows the research tree obtained in thisSMS. The percentage of included papers in each topic is shown below the topic. It should be noted that in theprocess of determining the subject of papers, thematic similarities are considered as Table 20. In Table 20, in eachrow, each topic in the topic column is equivalent with words inserted in the similar concept column.Table 21 shows the compounds obtained as a result of the co-occurrence matrix and the topics obtained as thelevel-one topics of the research tree. Based on Section 8, one of the rubrics applied for evaluation of our SMS , Vol. 1, No. 1, Article . Publication date: February 2021. loud Broker: A Systematic Mapping Study • 43
Table 21. The topics obtained as the level-one topics of the research tree
NO Topic Integrated Keywords1
Service Discovery DiscoveryCloud servicePrediction Service Allocation Scheduling/Provisioning Energy Management Virtualization/Green Computing Service Selection Service Selection/Graph-based Methods SLA management SLA/Trust Resource Allocation Resource Allocation/IaaSResource Management/Data Center Pricing Pricing/GameWeb Service/Cloud Market Monitoring Cloud Provider/Monitoring Recommendation Recommendation Other Particle Swarm Optimization/Cloud ManufacturingMiddleware/SaaSIoT/Genetic AlgorithmDistributed Systems/Learning Service Composition and Integration Ontology/Optimization/Service Composition/QoSIntegrationis threats to validity. In the validation process, the primary goal is providing some evidence for resolving allexisting threats facing our SMS. In the following, the prominent evidence is investigated and discussed. • Obtaining a set of high-quality studies: for obtaining a complete set of high-quality studies in the fieldof the cloud broker, a complete procedure is designed as a search strategy that comprises advantages ofboth famous search methods i.e. SLR and SMS. Therefore, we believe that our review is reliable. • Obtaining the most related studies: one of the most primary advantages in our search strategy is gradualevolution of keywords set during conducting the SMS. In should be stated that in some situation, somekeywords in the set of keywords did not convey a concept of the cloud broker. Accordingly, all suchkeywords in a category by applying some logical operators (AND, OR) have been merged. Examples ofmerging some keywords are provided in
𝑆𝑢𝑝𝑝𝐹𝑖𝑙𝑒 𝐸 ,𝑇 . • Reviewer’s biases or misunderstandings during the process of study selection:
To prevent thesechallenges in our SMS, at first, the selection process is independently done by two reviewers, then thepossible disagreements were solved by the third reviewer and decision rules (see Section 7.5). • Creating some forms to extract raw-data: as another threat, during the execution of the data extractionphase, some included studies were without author’s keywords and we extracted suitable keywords forthose studies considering the context of it and stored the extracted keywords in the related forms. • Assigning proper name for each level-one topic:
After clustering all keywords (extracted and gener-ated), we assign a proper name for each cluster that describes the concept of that cluster. For example, forthe
Resource Allocation topic, there were some keywords like
Resource Allocation , Resource Management ,and
VM Scheduling . However, we pick out
Resource Allocation as a suitable name for the topic, because webelieve this name can goodly convey the desired concept of studies. Nevertheless, in the presented SMS,in addition to both considering semantic and syntax of keywords for each topic, we also brought up theselected names between our team to remove the potential bias.as a suitable name for the topic, because webelieve this name can goodly convey the desired concept of studies. Nevertheless, in the presented SMS,in addition to both considering semantic and syntax of keywords for each topic, we also brought up theselected names between our team to remove the potential bias.