An XR rapid prototyping framework for interoperability across the reality spectrum
AAn XR rapid prototyping framework for interoperability across the reality spectrum
Efstratios Geronikolakis , George Papagiannakis Foundation for Research and Technology Hellas, 100 N. Plastira Street, 70013 Heraklion, Greece Computer Science Department, University of Crete, Voutes Campus, 70013 Heraklion, Greece
Abstract
Applications of the Extended Reality (XR) spec-trum, a superset of Mixed, Augmented and Virtual Reality, are gaining prominence and can be employed in a variety of areas, such as virtual museums. Examples can be found in the areas of education, cultural heritage, health/treatment, entertainment, marketing, and more. The majority of com-puter graphics applications nowadays are used to operate only in one of the above realities. The lack of applications across the XR spectrum is a real shortcoming. There are many advantages resulting from this problem's solution. Firstly, releasing an application across the XR spectrum could contribute in discovering its most suitable reality. Moreover, an application could be more immersive within a particular reality, depending on its context. Furthermore, its availability increases to a broader range of users. For in-stance, if an application is released both in Virtual and Augmented Reality, it is accessible to users that may lack the possession of a VR headset, but not of a mobile AR de-vice. The question that arises at this point, would be "Is it possible for a full s/w application stack to be converted across XR without sacrificing UI/UX in a semi-automatic way?". It may be quite difficult, depending on the architec-ture and application implementation. Most companies nowadays support only one reality, due to their lack of UI/UX software architecture or resources to support the complete XR spectrum. In this work, we present an "auto-matic reality transition" in the context of virtual museum applications. We propose a development framework, which will automatically allow this XR transition. This framework transforms any XR project into different realities such as Augmented or Virtual. It also reduces the development time while increasing the XR availability of 3D applications, en-couraging developers to release applications across the XR spectrum.
Keywords: Virtual Reality, Augmented Reality, Mixed Reality, Extended Reality, Cultural Heritage, Virtual Museums
Throughout the years, computer graphics applications are becoming more and more prevalent each day. People use them for a variety of purposes, for instance, entertainment [23], education [31], training [19], even research [19]. A crucial term mainly mentioned in this work is XR. Ex-tended Reality (XR) is a fusion of all the realities – including Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) – which consists of technology- mediated experiences enabled via a wide spectrum of hard-ware and software, including sensory interfaces, applications, and infrastructures. XR is often referred to as immersive video content, enhanced media experiences, as well as interactive and multi-dimensional human experienc-es. [30] The number of 3D applications that exist across the XR spectrum is considerably limited. Discovering a Virtual Re-ality application for instance, that exists in Augmented Reality as well, is a rare situation. It is comprehensible, con-sidering that the two previously mentioned realities greatly differ in terms of user experience (interaction). As a result, the amount of effort needed for transitioning such an appli-cation from Virtual Reality to Augmented Reality or vice-versa (across the XR spectrum) would be approximately identical to that of developing it from scratch. This fact is the primary reason why developers do not consider releasing an application across the XR spectrum. It is quite unfortu-nate since there is always a chance for it to be more appealing in another reality than the one it was initially de-signed for. Depending on the application content and its general pur-pose, it might be more suitable for Virtual Reality than Augmented Reality, for instance. Such an example would be surgical training applications [20]. In such cases, we need the users to fully immerse themselves and try to remember the steps and movements (with the controllers) they execut-ed inside the virtual world. That would not be possible for a mobile device with Augmented Reality. The previously mentioned problems would be addressed if the transitioning procedure across the XR spectrum was quick and could be executed effortlessly. Developers would quickly port their application through the XR spectrum and discover the most appropriate case for each application. This problem can be addressed by developing a framework, able to be used through the game engine, which would automati-cally transit the application project across the XR spectrum (generating a ready-to-operate project), while respecting the developers’ choices. This work aims to relieve the developers (and especially new ones) from a burden they regularly face: the transition of a computer graphics application across XR (from VR to AR, for instance) and more specifically, virtual museum ones. This transition usually requires many actions from the developers’ side. Of course, tutorials that exist online only explain how to start an application in the chosen reality from scratch, as shown in [2]. Similar are the cases for the other platforms/realities as well. Our tool aims to relieve devel-opers from this confusion and offer them a swift solution to their problem, which will be an operating project for the reality of their choice, for their application. Another intention of this work is to reveal the way that a cultural heritage application can transcend across XR, (meaning adapting the needs of the application to the new reality/device requirements) by using our framework. It solves the problem above and is a part of the results of this work. An example of such application XR transition is de-scribed in [7], where an application transits from Virtual Reality to Augmented Reality manually. Our framework automates an important part of this process. We split this work into five sections. Broadly, this disserta-tion's main roadmap is the introduction, previous work, the definition of the problem, and its solution alongside its eval-uation. In the second section, we mention some other inspiring cul-tural heritage applications and some authoring tools that contribute to the development of such applications. Follow-ing that, we discuss some applications that exist across XR, which is what we designed our tool to do automatically. Then, we cite other XR applications, for which their primary purpose was to educate the users (serious games). In the third section, we introduce the essential elements of each of the platforms and devices used for this work. After-wards, we declare the definition of the principal problem of this work. Then, we present our framework, the "XR transi-tion manager", which performs a fast and easy transition across XR for an application. In the fourth section, we pre-sent our framework's evaluation scheme, along with the results, and we draw some conclusions. Lastly, in the fifth section, we discuss the future work we could do, in order to improve our tool further.
There are several types of work that we will present in this section. Most of them concern the preservation of Cultural Heritage because our framework is targeted towards the transition of virtual museum applications across XR. We will present some exciting state of the art virtual museum applications. As our framework will also involve the ability of transition across XR, we will examine some existing au-thoring tools and applications that work in various realities.
Virtual museums
Nowadays, virtual museums have become more and more prevalent. The idea of designing a virtual version of a real museum is a fascinating one, as it is a way to contribute to cultural heritage preservation. That is because, by generating a virtual museum application and distributing it to people, they can gain access to the virtual version of the particular museum from anywhere in the world. It is a very innovative idea since every person that maintains a mobile device will virtually visit the museum of their desire, without having to travel directly to it, making their life more convenient. A rather extensive study [18] presents different Mixed Reality technologies from various researchers regarding the area of virtual museums.
Holographic Virtual Museums
Since the advancement of holographic technology, AR headsets are evolving, including interactive features like gesture and voice recognition and improvements on resolu-tion and FOV. Besides, untethered AR headsets paved the way for mobile experiences without external processing power from a PC. Such embedded systems facilitate excel-lent tools to represent virtual museums [14] due to their lack of cables and enhanced interactive capabilities. Virtual Mu-seums are institutional centers in society's service, open to the public for acquiring and exhibiting the tangible and in-tangible heritage of humanity for education, study, and enjoyment. Also, True Augmented Reality has recently been defined as a modification of the user’s perception of their surroundings that the user cannot detect [24] due to their realism. Virtual characters and objects should blend with their surroundings, attaining the “suspension of disbelief”. Many approaches on holographic cultural heritage applica-tions emerged in modern years, each one concentrating on a different aspect of representing the holographic exhibits within the real environment. A published survey [12] inves-tigated the impact of Virtual and Augmented Reality on museums' overall visitor experience, highlighting the social presence of AR environments. [17] presented a correlation of the latest methods for the rapid reconstruction of real hu-mans using as input RGB and RGB-D images. They also propose a complete pipeline to compose highly realistic re-constructions of virtual characters and digital asserts suitable for VR and AR applications. Storytelling, Presence, and Gamification are three critical fields that should be consid-ered when creating an XR application for cultural heritage. [18] presented a comparison of existing MR methods for virtual museums and pointed out the importance of these three fields for applications that contribute to cultural herit-age preservation [11]. Furthermore, in [10], fundamental elements for MR applications alongside examples are pre-sented. Another recent example [31] introduced two Mixed Reality Serious Games in VR and AR, comparing the two technolo-gies over their capabilities and design principles. Both applications showcased Knossos's ancient palace in Minoan Crete, Greece, through interactive mini-games and a virtu-al/holographic tour of the archaeological site using Meta AR glasses. [1] successfully published an AR application for visualizing restored ancient artefacts based on an algorithm that addresses geometric constraints of fragments to rebuild the object from the available parts. Authoring tools for content creation in MR
Content is an essential part of a 3D application (if not the most crucial one). Extensive care must be taken during con-tent development, as it constitutes the application and user experience base. Since creating content is a time-consuming and challenging procedure, different authoring tools for con-tent creation have come to the surface to ease developers' lives. In this section, we present some of the prevailing works in this field.
Platforms for Gamified Content Creation
Authoring tools and additional content creation platforms emerged in modern years to fulfill the demand for interac-tive MR applications. RadEd [21] highlights a new web-based teaching framework with an integrated smart editor to create case-based exercises for image interaction, such as taking measurements, attaching labels, and selecting specific parts of the image. It facilitates a framework as an additional tool in complex training courses like radiology. BricklAyeR [26] is a collaborative platform designed for users with lim-ited programming skills that allows the creation of Intelligent Environments within a building-block interface. In [3], the authors present tools for interactive virtual human population in the concepts of cultural heritage. These tools utilize open source generic frameworks for generating inter-active 3D virtual environments and content. ExProtoVAR [22] is a lightweight tool to produce interactive virtual pro-totypes of AR applications designed for non-programmers lacking AR interfaces experience. ARTIST [13] is a plat-form, which provides methods and tools for real-time interaction between human and non-human characters to generate reusable, low cost, and optimized MR experiences. It aims to develop a code-free system to deploy and imple-ment MR content while using data from heterogeneous resources semantically. The aforementioned solutions pro-vide developing environments to generate MR experiences. However, they lack advanced authoring tools and educa-tional curriculum to support advanced educational - training scenarios. Lastly, in [33], the authors propose a gamified way of content creation for a training application, through a user interface by connecting blocks of events or setting up the desired events through Virtual Reality.
Unity MARS
Unity MARS [28] is a novel AR authoring tool, produced by Unity3D in 2020. It is unique since it offers the ability to develop and test AR applications from the Unity MARS environment without building the application each time. With the use of proxies, which are 3D objects imitating real-world objects, and “fuzzy authoring” developers can define the minimum and maximum measurements for them rather than code precise values. Moreover, with a relatively simple drag-and-drop feature, developers can place their 3D models in the scene, and Unity MARS produces all the appropriate proxies and conditions for them. It also supports different kinds of real-world data, such as images and surfaces. In the near future, it will also support body tracking. It is a fasci-nating and time-saving authoring tool for AR that aims to speed up AR development.
Applications existing across XR
Following a rather extensive search for previous works on applications existing across XR, we did not find any scien-tific results. It is acceptable to the perspective that it is rather challenging for an application to exist across XR. From our experience, we believe that one of the reasons for this ab-sence of examples could be the porting complexity. Of course, it is a procedure, which is not unachievable. It is doable, but since it is a time-consuming procedure [9], many people refrain from commencing it. They prefer generating new content for a particular platform and push on. They do not ponder porting the same applications to other platforms, since they may consider this “recycling” of the same appli-cation. All the previously mentioned thoughts are based on our opinion, of course. Another reason could be that apart from the challenge of the porting procedure, each platform has its specifications and requirements. As a result, an application in Virtual Reality that uses controllers would need total rework to operate in a mobile device and Augmented Reality. Mobile devices do not encourage the use of controllers. Their primary input device consists of a touchscreen. This fact completely changes the whole user experience, and it is reasonable to refrain the developers from attempting the port. Multiple applications have certain principles and require a specific structure in order to offer their full experience to users. Suppose one of these principles is absent (in our ex-ample, the fact that users should be able to move their hands around freely and see them in the application). In that case, the immersion and, as a result, the user experience drops significantly. We managed to spot one example, nevertheless. That would be a Virtual Reality application, which exists in Mixed Real-ity as well (HoloLens). Within this application, users perform a surgical operation, named Total Knee Arthroplas-ty. More specifically, users follow a sequence of actions, visualized by holograms, to complete the operation. We found out that an approach to transfer this application to HoloLens exists. However, it remained incomplete (it does not contain the whole operation), probably because of the reasons we explained.
Figure 1.
Total Knee Arthroplasty in Virtual Reality (left) and Mixed Reality (right).
Mixed reality serious games for smart education
Considering we consumed a large amount of time to pro-duce our virtual museum application and worked to make it available for multiple platforms, we scrutinized for work regarding Mixed Reality serious games. We performed this research to determine the gamification methods that were used by researchers in their respective works, to collect ide-as for our virtual museum. In [32], the authors designed an application related to Knos-sos archaeological site. This application intends to educate users concerning the history of Knossos through a series of mini-games. They created several versions of this applica-tion across XR. They perceived that the same application could not be applied across XR because each one had its boundaries and characteristics. In [15], they clearly define the meaning and significance of serious games and gamifi-cation while presenting different technologies for immersive heritage applications. Another work, bearing valuable in-formation for constructing Mixed Reality applications for cultural heritage, is specified in [5]. Similarly, in [16], the authors present an inspiring, serious game application for intangible cultural heritage and, more explicitly, dancing. Users can learn many traditional dances by following a se-quence of movements, while the software provides them with feedback regarding how well they performed these movements.
Each XR application has different essential elements regard-ing the camera and user interaction functionality. These elements vary depending on the platform and hardware that the application operates. The platforms/operating systems that most 3D applications use (in Unity3D [27]), and we studied in this work are four. The first one is “PC, Mac & Linux Standalone”, for applications that operate on a desk-top computer. The second one is the “Android” platform for applications that operate on a mobile device or headset that supports android. Next is “iOS” for mobile devices that run Apple’s iOS and, lastly, “Universal Windows Platform” for mobile devices utilizing Windows. The hardware on which most 3D applications operate is either VR Head Mounted Displays (HMDs) or mobile devices (smartphones, tablets, and more).
Virtual Reality Applications
Virtual Reality technology generates an artificial environ-ment that immerses the users, causing them to believe that they are a part of it, doing there and being there. Although this technology has emerged long ago, it is in the latest years that it made its first steps in the market and became well-known. For the users to enter the virtual world that a VR application provides, they have to wear a VR head-mounted display (HMD), which precludes them from having access to the real world, as long as they wear it. As a result, users experience full immersion. This technology is used widely for many purposes, from entertainment [23] to training [19] and research [19].
Augmented Reality Applications
Augmented Reality technology merges the real with the virtual world. Applications made utilizing this technology usually operate on mobile devices containing at least one camera component. A camera is profoundly needed because this is the users' “window” to the virtual world, where the virtual 3D objects will reside. Augmented Reality offers partial immersion since users still have access to the real world. Both iOS and Android mobile devices are eligible for operating Augmented Reality applications. Each platform has produced its version of Augmented Reality Software Development Kit (SDK), namely ARKit [4] and ARCore [8].
Holographic Augmented Reality
Holographic Augmented Reality, as the name implies, is very close to straightforward Augmented Reality. It differs in the way that all virtual objects are rendered as holograms. With Holographic Augmented Reality’s aid, users can ob-serve these holograms through a special HMD that highly increases realism. As a result, not only do they experience the presence of virtual objects being in their room, but they can also interact with them using hand gestures. Thus, Hol-ographic Augmented Reality is bound to partial immersion, since users have access to the real world, but in this case, realism is even higher compared to straightforward Aug-mented Reality.
All-in-one Unity XR transition manager
Transitioning across XR, or even setting up a new project for the first time may be time-consuming for developers [9]. For instance, assume that a developer desires to start pro-ducing an Augmented Reality application for Android mobile devices. After starting Unity, they will have to pon-der some crucial elements: 1.
Is there a specific Software Development Kit (SDK) for this project that could be useful? 2.
Where can this SDK be downloaded? 3.
Is it open-source/free? 4.
Which version is the most appropriate? (Mostly, the latest versions are the best choice, but there are also some other situations like testing or experi-menting, where an older SDK version is required.) These are the main questions that cross the developer's mind. When the developer finishes thinking about the above, what they need to do is probably open a web page and begin searching one-by-one the above questions to at-tain the required answers. Sometimes this could be a relatively fast procedure (especially if the developer has much experience). However, in the case of newcomers de- velopers, this could be quite a time-consuming procedure [9]. It is where we can prove our work to be beneficial both to experienced and inexperienced developers. We describe the first part of it below. XR transition download manager
Another question that arises is, “What if the SDK searching and downloading procedure took place from inside Unity?”. It is why we created an SDK download manager, which is accessible through the game engine.
The basic architecture of XR Transition manag-er
The “XR Transition manager” has a simple architecture. Its primary function is to connect to the main hosting website of the desired SDK. If the manager detects the requested version, it downloads the selected SDK through that connec-tion and then follows the XR transition procedure. The manager handles the camera component and sets it up cor-rectly. Developers need to specify the game-object of the camera, in any case. Overall, the manager connects with the following four components: 1.
Websites hosting the SDKs. 2.
Downloaded SDKs. 3.
Unity scene. 4.
Main camera component of the scene. We can view the central architecture of the manager in Fig-ure 2 below.
Figure 2.
The basic architecture of the XR Transition manager.
The main code structure and logic of XR Tran-sition Manager “XR Transition Manager” consists of a combination of edi-tor scripts. We designed them using the C
Menu Script This collection consists of one script only, namely Reali-tiesMenu. This script is the backbone of the whole framework. It is responsible for calling and invoking the function that developers select when they click an option from the respective Unity3D menu. For instance, if develop-ers navigate to Realities menu and click the “ARCore” option under Switch Reality/Augmented Reality, the base script is responsible for calling the function, which contains all the necessary steps to perform an XR transition. 3.5.2.2
SDK Download & Setup Scripts In this collection, the SDK downloading and installing scripts exist. There are four scripts in this category, one for each of the currently supported SDKs. There is one for AR-Kit (DownloadARKitWindow), one for ARCore (DownloadARCoreWindow), one for Mixed Reality (Down-loadMixedRealityWindow), and another one for Oculus Integration (DownloadOculusWindow). All these scripts have the word “Window” appended to their name, consider-ing the very first thing they do is to enable a window, in which developers will be able to select their desired version to install. In addition, they can check whether they would like the downloaded SDK to be installed immediately after downloading finishes from this window. These scripts accommodate an OnGUI function, which con-tains everything drawn to the window from which developers select the SDK version that interests them. As its name informs us, it contains everything that exists On the Graphical User Interface.
Figure 3.
The contents of the OnGUI function with their respective matches to the window that the developer sees. (Each color on the left matches with its respective on the right.)
The OnGUI function follows the Download function, where the main downloading procedure takes place. Within it, there are some hardcoded links, each representing a version of the preferred SDK. Depending on the developers' SDK version decision, the script selects the appropriate link. The downloading procedure begins once the developers' internet connection is verified; otherwise, the script prints a repre-sentative error message. During downloading, we calculate and display the progress using a progress bar window. We also give developers an option to cancel the download pro-cedure by clicking the “Cancel” button located at the window's underside. When downloading terminates, we save the downloaded data inside the project assets folder. Then, the “Download” function proceeds by checking whether developers chose to install the SDK instantly or not. If they chose to do so, it imports the package, using the default package importing method of Unity3D. However, the same does not hold for ARKit, where the SDK comes in compressed zip files rather than unitypackage files. For ARKit, we take extra steps to decompress the file and install it, since Unity3D does not offer a built-in system to decompress zip files automatically. We decided to use a third-party decompression tool, termed UniZip [29], which handles the specific procedure. Once the installation step finishes, the definitions section takes place. We take particular precautions for scripts containing code and expressions targeting specific SDKs because they will not compile unless these SDKs exist in the project. For in-stance, scripts that belong to the “XR Transition Scripts” collection include specific commands and utilize prefabs existing in the SDKs they represent. If these SDKs are not present during the compilation phase, it will result in compi-lation errors, prohibiting the whole project from operating. Thus, we created some specific definitions (one for each SDK) for the framework to identify the currently installed SDK each time. Of course, this applies only to SDKs that users installed through our framework. Using these defini-tions, the code that will be compiled each time will be the one that represents the currently installed SDK, thus solving the compilation problem for SDK specific code.
Figure 4.
An example of managing definitions for the case of ARKit.
XR Transition Scripts “XR Transition scripts” is the final collection of the “XR Transition manager”. In this collection, we added four scripts, each of them being responsible for transitioning to one of the four supported platforms. Once again, the OnGUI function exists in each, setting up the window, where the developers select the main camera. Once developers press the proceed button, the script applies all the required options in the player settings menu. Then it performs a platform switch to the desired platform, constructs, and instantiates all necessary prefabs to generate a scene, which will operate successfully for the desired platform. Of course, we take some extra precautions to ensure that the manager will con-tinue operating even if something does not go as intended. For instance, it does not take for granted that the correct SDK is installed even in this case. On the contrary, it tries to make sure that everything regarding the SDK is installed and exist. Otherwise, it interrupts the procedure by notifying the developers through an error message.
Downloading and installing SDKs through the XR Transition Manager
Our manager consists of editor scripts, written in C
Importance of the XR Transition SDKs download manager
As we mentioned previously, the download manager could be handy for developers while starting a new project. It speeds up the procedure and makes it exceptionally effort-less. Also, it increases productivity, since it reduces the context switch between the current workspace and browsing [25] that developers may result to in case they do not re-member all the necessary settings for the platform of their choice. Without our manager, developers, apart from search-ing and downloading the desired SDK on their own, would also have to work further to import it (unzipping the SDK and placing it in the right place). Our download manager handles this likewise. Additionally, it can be useful when developers would like to transit their application across XR. It is able to download the new SDK and get the developers up and running to develop their application in no time.
XR Transition Feature
Our “XR Transition manager” includes another fundamental and critical functionality. It downloads whichever supported AR/VR/MR SDK needed, which saves a substantial amount of developers’ time. However, even though an SDK is downloaded and installed, further work is required from the developers’ side for it to work correctly for the final applica-tion to be built and operate flawlessly. For instance, to build one that uses Google’s ARCore, the following options must be set: The target platform must be Android. The default Graphics API must be either OPENGLES2 or OPENGLES3. The packages “AR Foundation”, “Multiplayer HLAPI” and “XR Legacy Input Helper” must be installed in the current project. The minimum Android version must be 7.0 (API Level 24). The Unity setting “ARCore supported” must be en-abled. Search in the SDK folders to find the specific cam-era default prefab and place it in the scene. Developers must memorize all the above to set the project configuration correctly and build a sample application. It can be challenging since all these steps contain specific de-tails (such as package names and settings) that are hard to retain. Especially for new developers, this can be rather frus-trating. Usually, developers open a web browser and begin searching to find the previously mentioned settings. If they are fortunate enough, they find them in a few seconds, but there is a possibility that the correct website will not appear to them immediately. It is why our manager includes an “XR Transition” feature. Once developers have downloaded and installed an SDK through our manager, they can decide to perform XR transi-tion through Realities menu. Specifically, by pressing the Realities button and then the Switch Reality one, they can select the reality of their desire (Fig. 9). When they define it, another window appears, prompting them to define their main camera game-object present in their scene (Fig. 10). In a 3D application, a camera object is mandatory for the play-er/user to view the scene and navigate. Thus, developers have to locate their main camera object in the Unity scene and place it in the appropriate box located in the window mentioned before. Once they do so, they have to click the “Proceed” button, and the SDK manager takes care of eve-rything else. Specifically, it switches to the appropriate target platform, installs the required packages, sets up the project correctly, and instantiates the appropriate camera prefab in the scene, in the same position and rotation as the previous camera object. The latter is deactivated since it is “replaced” by the new one, but it remains in the scene, in case developers would like to do something else with it, or save it if they would like to return to the previous reality. This procedure exists in our SDK manager for switching to Augmented Reality (Android and iOS), Virtual Reality (Mobile VR – Oculus Go/Quest), and Mixed Reality (Mi-crosoft HoloLens).
Using the XR Transition Manager “XR Transition Manager” is a framework for Unity3D that aims to speed up the developing process of 3D applications. In section 3.11, we described our framework's primary tech-nology. In the next sections, we will present some basic examples regarding its usage.
Downloading and installing an SDK
Once developers import all the required manager scripts correctly in their project, a menu “Realities” should appear in the menu bar of Unity3D. This menu contains four op-tions. The first one is “Software Development Kits” and provides another four should the developers hover the mouse over it. These options are “Download ARCore”, “Download ARKit”, “Download Oculus Integration” and “Download Mixed Reality Toolkit”.
Figure 5.
The supported SDKs, as they are presented in the "XR Tran-sition Manager" menu.
Currently, our manager supports these four SDKs. By se-lecting one of these, developers can download the corresponding SDK. In this section, we will highlight the case of ARCore. However, the same procedure applies to the rest of the SDKs. When we click the “Download ARCore” option, the following window appears.
Figure 6.
Choosing the version of ARCore and its installation before downloading it.
This window provides developers with the following op-tions: Version:
Developers have the option to select the SDK version of their choice. The versions are cur-rently hardcoded in the manager. Therefore, to add newer versions, a new version of our manager has to be released as well each time. Install package:
If developers check this button, the SDK is imported and installed automatically af-ter download finishes. Otherwise, either a .unitypackage or a .zip file will appear in the pro-ject path for the developer to install/import it manually.
When developers set the version and the package install option, the procedure continues by clicking the “Proceed” button.
Figure 7.
The progress of SDK downloading.
The window depicted above indicates the progress of the download procedure. Developers have also the option to stop it at any time by clicking the cancel button. In case of success, the following window appears, indicating that the download procedure was successful.
Figure 8.
Success message for the successful download of SDK.
After developers click the OK button, depending on their initial choice, the SDK will be imported and installed, or they will receive a .unitypackage file containing the SDK, to install it by themselves.
Switching reality
The second option that “Realities” menu provides is transi-tioning the project across XR depending on the developers’ choice. Again, in this example, we will focus on ARCore, but the same procedure applies across XR. As visible from the image below, there are three options under the “Switch Reality” tab.
Figure 9.
The supported realities of switching to, depending on the se-lected SDK/platform. Augmented Reality: This reality is mostly for mo-bile devices equipped with a camera. It is because, as mentioned previously, Augmented Reality offers partial immersion. For Augmented Reality, the two sub-options that are currently supported by our manager are ARCore and ARKit. Virtual Reality: This reality is for an application operating on a Head-Mounted-Display (HMD). Currently, our manager supports only Oculus Inte-gration, rendering the supported devices to be mobile VR and Oculus Go/Quest. Mixed Reality: This reality is for VR/AR applica-tions operating on Microsoft devices. Currently, we tested this option on Microsoft HoloLens (Holo-graphic Augmented Reality). Developers should consider switching to a scene supporting the SDK that they have downloaded. If they have not down-loaded the correct SDK, an error message will notify them about it, and the manager will prompt them to download it. When developers select the reality, they would like their project to switch into, the following window appears.
Figure 10.
Settings window of switching to Augmented Reality for ARCore/Android.
The next step is to specify the main camera of their applica-tion, which serves as the users' eyes. Once it is set and developers click the “Proceed” button, the procedure of XR transitioning begins. The time developers have to wait af-terward depends on the size of their project. This action performs a platform switch, sets all the required settings for the current SDK, and adds the main prefabs/objects in the scene required by the SDK for a successful and working build. Of course, after the operation finishes, developers can add/delete SDK related prefabs according to their desire. The XR transitioning procedure performs all the necessary options in a few seconds; otherwise, the developers would have to do it manually.
Uninstalling an SDK
The uninstalling feature could not have been missing from our SDK manager. There is a separate option in the “Reali-ties” menu regarding uninstalling an already downloaded SDK. The image below depicts it. Figure 11.
The SDK uninstall feature.
Someone could submit the following question. “Why should a developer do that through the ‘Realities’ menu and not by just selecting the SDK folder and manually deleting it?”. That would be the same as deleting an unwanted program's folder in our computers. The program would not be there, but a lot of information and data would remain internally. The same holds for our SDK manager. Developers may de-lete the SDK folder separately, but all the options and settings our manager performed during this SDK installation would remain. These additional settings reset by our manag-er are the main reason for developers to have a “healthy” and “ready to operate” project after the SDK removal.
Automatic reality working camera
Sometimes, developers would like to place a working cam-era on their current project and reality, without needing to undergo an XR transitioning procedure (testing purposes, for instance). Our manager handles this case since it offers an option to spawn an XR specific camera on demand. We call this option “Spawn Current Reality Camera”. When selected, our manager automatically detects the platform and installed SDK developers are working on and adds the cor-responding test-ready camera to the scene.
Development of XR Transition Manager
The development procedure of the “XR Transition Manag-er” was relatively smooth, with a few challenges that took place along the way.
The SDK handling of XR Transition Manager
The “XR Transition Manager’s” central idea is to host sev-eral valid download links to each version of various SDKs and provide functionalities of installing and setting them up. It aims to ease the developers to quickly develop and build their applications without any SDK related errors. For each SDK, we include links to its stable versions in the code used for downloading it, in case the developers select the specific version. We use the UnityWebRequest API to validate the connection and proceed to the actual download. When it finishes, the downloaded file is recorded temporarily in the assets folder to be imported and installed to the project. We import the file using the ImportPackage utility of Unity. Unity offers the ability to make a prompt appear to develop-ers when importing a package, enumerating the package contents to specify which components to install. We decided to disable this prompt since we install only the SDK's man-datory components (thus, everything is needed) to save time during installation and importing. After these procedures end, we delete the temporary downloaded file, leaving only the installed components.
Challenges Faced During Development
The main challenges that we faced during the development of the “XR Transition Manager” were two: Firstly, Apple’s ARKit SDK is distributed in .zip form, in-stead of .unitypackage. However, the ImportPackage utility of Unity would not be successful in installing such files. To overcome this issue, we had to install Apple’s ARKit with-out using the utilities of Unity. We investigated different approaches by writing our C Results and Conclusions
In this section, we will summarize our work. It was a fasci-nating topic, which we enjoyed working on. Since there is no actual limit to what can be done or added to this work to make it even more helpful and meaningful to the developers, we will discuss some potential ideas to be added in the fu-ture.
Summary
In this work, we initially presented a framework for manual-ly transitioning a cultural heritage application across XR (from VR to holographic AR). We presented how we trans- ferred a Virtual Reality application to holographic Aug-mented Reality and the challenges we faced during this platform switch. Afterward, we introduced the main prob-lem with application transitioning across XR in a 3D application and discussed why this should occur automati-cally. We then described “XR Transition Manager” in detail, its functions, and how it works. Belatedly, yet importantly, we showcased our latest work on Cultural Heritage, an ap-plication for the Industrial Museum and Cultural Heritage of Thessaloniki [6], and viewed the results of our manager tested on this application. Evaluation
To examine our system's overall user experience, we con-ducted a preliminary user-based evaluation with ten users (eight of them were proficient Unity3D developers, whereas the other two were relatively new to Unity3D). The main research questions were the following: -
How time-consuming is the manual SDK installa-tion/reality switch/SDK uninstallation for a project in Unity3D? -
Is the SDK download function of our system pref-erable in comparison to the manual downloading procedure? -
Is the “Switch Reality” feature of our system pref-erable in comparison to the manual platform switch and settings application procedure? -
Is the SDK Uninstallation feature of our system preferable in comparison to the manual deletion of each SDK? -
Is the “Spawn Current Reality Camera” function of our manager useful? -
Would a developer prefer to use our manager for their Unity3D projects?
Methodology and participants
We divided the experiment into four different parts, each for one of the questions previously described. It is worth noting that all the participants were software developers, and they were familiar with Unity3D. 4.2.1.1
Part 1: Manual Installation of ARCore SDK In this part, we asked the participants to manually download and install an SDK (specifically ARCore SDK). They had to do it manually at first. They had to search for tutorials (in case they did not know how to install it). To succeed in this part, they had to include all the necessary files and directo-ries of the SDK needed to compile successfully (thus, no compile errors in the Unity3D editor). 4.2.1.2
Part 2: XR Transition Testing In this part, we asked the participants to switch to the reality supported by the SDK they downloaded in the previous step. Again, they had to do it manually. To succeed in this step, they had to press the “Play” button of the Unity3D editor to run without any errors. For this part, we measured both the time they needed to initiate a platform switch manually and apply all the necessary settings. Finally, we received their feedback on the intricacy of the procedure. 4.2.1.3
Part 3: SDK Uninstallation In this part, we asked the participants to uninstall the SDK they installed in the previous steps. To succeed in this step, they had to reverse the Unity3D editor to its initial state (when they started the evaluation). They had to do it manu-ally. In the end, we measured the time they needed and received their feedback regarding the complexity of the pro-cedure. 4.2.1.4
Part 4: Performing the tasks with “XR Transition Manager” As this evaluation's final step, we asked the participants to perform the previous actions again, but this time, using our manager. This procedure was speedy (ten to twenty seconds) for users to find and click the appropriate buttons on the “Realities” menu. We should note that the SDK download time and system execution time for platform switch were not calculated, since that depends on the system and internet connection speed. In the end, we asked them to provide us with their feedback about the current procedure, and finally, we asked them if they would prefer to use our “XR Transi-tion Manager” in their future Unity3D projects. Results
Part 1: Manual Installation of ARCore SDK For this part, we measured the average time they needed to install ARCore SDK to their Unity3D project. We told them to search the internet freely for tutorials and steps on how to do this. For this part, time was our variable. It is important to note that we did not measure the time needed for down-loading the SDK for this step, or the SDK system installation procedure. We present the results for this part in the table below.
Table 2.
Participants' results for the first part of our evaluation.
Part 2: XR Transition Testing For this part, we also measured the average time the partici-pants needed to manually perform a transition across XR and prepare a sample scene of ARCore to run in Unity3D editor. Again, the participants were free to search for tutori-als online on how to do this, in case they did not know. We did not measure the time needed for platform switch, since it depends on the system hardware. We present the results in the table below.
Participant Hours Minutes Seconds Time Decrease Percentage (for average time of 15 s)
Table 3.
Participants' results for the second part of our evaluation.
Part 3: SDK Uninstallation It is the final part where we measured time. In this part, we wrote down the time the participants needed to remove ARCore SDK from their Unity3D project and return the project to its initial state. Again, system-dependent time was not measured (platform switch). We present the results in the following table.
Participant Hours Minutes Seconds Time Decrease Percentage (for average time of 15 s)
Table 4.
Participants' results for the third part of our evaluation.
Part 4: Performing the tasks with “XR Transition Manager”. After participants finished doing the tasks manually, we requested to perform them again using the “XR Transition Manager”. It was a relatively quick procedure because they execute these actions with a button press. So that would be a few seconds (ten to twenty) for each participant. Finally, we asked the participants to give us their feedback by answering some questions.
Discussion
The current evaluation brought us fascinating results. Some participants were quick enough for both installation and transitioning across XR as well, whereas some participants took their time (about twenty minutes). For a programmer's development, twenty minutes is quite a long time for such a procedure, which our manager seeks to avoid. The same applies to the XR transition procedure as well (twenty-two minutes was the longest). The uninstallation procedure was quicker, although some cases needed some time (five or seven minutes, for instance, is considered quite long for such a case).
Participant Hours Minutes Seconds Time Decrease Percentage (for average time of 15 s) Then, participants tried our manager. The amount of time needed for all these tasks was between ten to twenty seconds per task for everyone. By taking the average of this time (15 seconds), we are able to calculate a percentage of saved time for each participant. We calculated the time decrease percentage (in seconds) for each participant and for each evaluation step. We calculated each percentage using the following formula: “Manual Time” stands for the time (in seconds) that partici-pants needed to do a requested action without using our framework. “Average Transition Framework Time” stands for the average time needed for participants to do an action by using our framework. This “time” is equal to 15 seconds, the average of a ten to twenty seconds window that each participant needed. These results are vital because our framework managed to save between 62.50% (worst case) and 98.89% of the participants’ time (best case). Conclu-sively, our framework saved more than 62.50% of the participants’ time (above average), which is clearly a win for us. Our current aim is to raise the worst-case percentage even more, in order to benefit even more developers. Apart from the time-consuming scenario, we also wished to know our participants’ thoughts about our manager and, more importantly, its ease of utilization. By observing Fig. 12, we understand that all participants gave a score of seven- plus out of ten for the SDK installing and XR transition tasks. For the uninstallation procedure, participants graded our manager with eight and nine out of ten. Two participants graded it with a six. We believe that is because our man-ager's SDK uninstallation procedure does not handle the cases where a plugin (.dll) might be used from Unity3D when users try to uninstall the SDK. This prevents our man-ager from entirely removing the SDK. It is a challenge for us to solve in the future. The majority of participants delivered a ten to our “Spawn Current Reality Camera function” . Two participants gave it a seven out of ten, and one participant gave it a six. Belated-ly, the participants seemed to approve our manager in general, by stating that they would use it in their Unity3D projects. They all gave a score of eight-plus out of ten when they were asked this question.
Evaluation Conclusion
To conclude the evaluation, the participants appeared to approve our framework, which made us more eager to up-grade it in the future continually. Our manager received more than a 60% score for each participant's feedback, which is quite a positive result. Besides, our manager saved more than 62.50% of their spent time. It is why we intend to work eagerly for the next months to stabilize it further and add more abilities that will be beneficial for developers. We present these additions in the following section.
Figure 12.
The results of the evaluation, depicted using graphs.
Future Work
We wanted to add some additions/features to our framework but could not due to the lack of time. We believe that these features would make the “XR Transition Manager” even more helpful to developers. We divided these features into the following sections.
Saving Time
Saving time is one of the most critical elements of our man-ager, if not the most prominent. It already saves a reasonable amount of time by managing four different SDKs and trans-cending across XR based on them. Until now, our manager successfully substitutes the project camera (which is selected by the developer). It adds all the necessary objects in the scene required by the current SDK to work correctly. It also applies all the necessary settings and installs the packages needed. However, we do nothing regarding the controls of the application. For instance, sup-pose we would like to transform an application operating on Oculus Go (mobile virtual reality) to run on a mobile device with Augmented Reality. Oculus Go has a controller, with which users can perform various interactions in their appli-cations. Suppose an application moves to Augmented Reality (from Virtual Reality) by utilizing our manager. In that case, users would not be able to perform any interac-tions since such mobile devices use completely different ways of interaction. For instance, most mobile devices use the touch screen to receive input from users. We want to provide our manager with a way to map all the supported functions from the input module of the originating reality to the target reality. For example, the pressing of a button in the Oculus Go controller could be mapped to a simple touch on a mobile device's touchscreen or a pinch gesture for Ho-loLens.
Expanding availability
We want to benefit as more developers as possible with this framework. At its current state, the number of developers who can profit is somehow limited. For developers to use this framework, they must be familiar with Unity3D, and their target device for their application must be a mobile device, Oculus Go, or Microsoft HoloLens. We want to put an end to this boundary by expanding the number of game engines on which our manager will be installed and the number of supported devices and SDKs. As a result, our foremost future objective is to make our manager available for the Unreal game engine since many developers use it. Moreover, we would like to provide support to all desktop Virtual Reality devices, SDKs, and mobile headsets/devices. That would significantly increase the number of developers who can be benefited by our manager. Last but not least, we would like to adjust the manager properly, like converting it to a single package or DLL file, to distribute way more effi-ciently.
Complete reality transformation
Our framework may currently transit a 3D application across XR, but it does not consider any theoretical elements that the target reality may have. We want to add this feature in the future. This feature will scan the application scene's contents and perform some necessary changes regarding the target reality. For instance, if we move an Augmented Reali-ty application containing a portal to Virtual Reality, that portal would be of no use. That is because Virtual Reality provides users with full immersion, and thus they do not have access to the real world. In that case, our framework should be able to detect such portals and disable them.
Smart Performance Adaptation
Our manager's main objective is to make the application porting procedure easier for developers, so that eventually, in the future, more and more 3D applications will be availa-ble for many devices. All those devices are different, though, in terms of hardware and overall performance. It is something that we would also like our manager to consider. We describe this in detail below. 5.1.4.1
Complexity of the models The 3D scenes of such projects contain different kinds and sizes of 3D models. Sometimes, 3D models tend to be very complicated because they might contain many vertices or complicated geometry. The more complex a model is, the less likely the application will operate smoothly on devices with lower specifications. That is a challenge that we would like to relieve the developers from. We plan to upgrade our manager to consider the device that the application will op-erate on and apply the necessary quality settings for the application to run smoothly on that device. We will also examine the case of applying reduction algorithms to those meshes, to diminish the geometry complexity and make them lighter for low specs devices. 5.1.4.2
Lighting One common solution to overcome low performance, con-cerning lighting, especially in low specs devices, is static lighting. It makes the scene less realistic but increases per-formance. We want to add some global illumination calculation algorithms to our manager. Again, depending on the target device, if the device is a low specs one, our man-ager would provide light and realistic real-time illumination algorithms. Even in those devices, real-time lighting will be available. We believe that such a feature will significantly improve the application quality while operating on low specs devices and render them highly realistic.
References [1]
Abate, A., Barra, S., Galeotafiore, G., Díaz, C., Aura, E., Sánchez, M., Mas, X., Vendrell Vidal, E. (2018). An Augment-ed Reality Mobile App for Museums: Virtual Restoration of a Plate of Glass: 7th International Conference, EuroMed 2018, Nicosia, Cyprus, October 29–November 3, 2018, Proceedings, Part I. 10.1007/978-3-030-01762-0_47. [2] ARCore – Quickstart for Android, https://developers.google.com/ar/develop/unity/quickstart-android, (last accessed July 26, 2020). [3] Arnold, D., Day, A., Glauert, J., Haegler, S., Jennings, V., Kev-elham, B., Laycock, R., Magnenat-Thalmann, N., Maïm, J., Maupu, D., Papagiannakis, G., Thalmann, D., Yersin, B., and Rodriguez-Echavarria, K., “Tools for Populating Cultural Herit-age Environments with Interactive Virtual Humans”, Open Digital Cultural Heritage Systems, EPOCH Final Event 2008, Rome, February 2008. [4] Apple’s ARKit, https://developer.apple.com/augmented-reality/arkit/, (last accessed Semptemer 2, 2020) [5] Bertuzzi, Juan, and Khaldoun Zreik. "Mixed Reality Games - Augmented Cultural Heritage." In Augmented Culture: Proceed- ings of the 15th Iberoamerican Congress of Digital Graphics, 304-307. SIGraDi. Santa Fe, Argentina, 2011. [6] Geronikolakis, E., Tsioumas, M., Bertrand, S., Loupas, A., Zikas, P., Papagiannakis, G., “New Cross/Augmented Reality Experiences for the Virtual Museums of the Future”. EuroMed (1) 2018: 518-527. [7] Geronikolakis, E., Zikas, P., Kateros, S., Lydatakis, N., Geor-giou, S., Kentros, M., Papagiannakis, G., “A True AR Authoring Tool for Interactive Virtual Museums”. CoRR abs/1909.09429 (2019). [8] Google’s ARCore, https://developers.google.com/ar, (last ac-cessed September 2, 2020) [9] How to port your app to another platform, https://mwdn.com/port-app-another-platform/, (last accessed September 2, 2020) [10] Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G., (Eds).: Mixed Reality and Gamification for Cultural Heritage, Springer, DOI: 10.1007/978-3-319-49607-8, (2017). [11] Ioannides, Marinos, et al., eds. Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection: 7th International Conference, EuroMed 2018, Nicosia, Cyprus, October 29–November 3, 2018, Proceedings. Vol. 11196. Springer, 2018. [12] Jung, Timothy & Tom Dieck, M. Claudia & Lee, Hyunae & Chung, Namho. (2016). Effects of Virtual Reality and Aug-mented Reality on Visitor Experiences in Museum. 10.1007/978-3-319-28231-2_45. [13] Kotis K. ARTIST - a reAl-time low-effoRt mulTi-entity Interac-tion System for creaTing re-usable and optimized MR experiences (2019). Research Ideas and Outcomes 5: e36464. https://doi.org/10.3897/rio.5.e36464. [14] Liarokapis, F., Sylaiou, S., Basu, A., Mourkoussis, N., White, M., Lister, P.F. An Interactive Visualisation Interface for Virtual Museums, Proc. of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, Eurographics As-sociation, Brussels, Belgium, 6-10 Dec, 47-56, 2004. [15] Liarokapis F., Petridis P., Andrews D., de Freitas S. (2017) Multimodal Serious Games Technologies for Cultural Heritage. In: Ioannides M., Magnenat-Thalmann N., Papagiannakis G. (eds) Mixed Reality and Gamification for Cultural Heritage. Springer, Cham. [16] Ott, Michela & Dagnino, Francesca & Pozzi, Francesca & Yil-maz, Erdal & Tsalakanidou, Filareti & Dimitropoulos, Kosmas & Nikos, Grammalidis. (2015). Serious Games to Support Learning of Rare 'Intangible' Cultural Expressions. [17] Papaeftymiou M., Kanakis E.M., Geronikolakis E., Nochos A., Zikas P., Papagiannakis G., “Rapid Reconstruction and Simula-tion of Real Characters in Mixed Reality Environments”, Digital Cultural Heritage Lecture Notes in Computer Science, Vol. 10605, 267-276, 2018. [18] Papagiannakis, G., Geronikolakis, E., Pateraki, M., Bendicho, V.M., Tsioumas, M. Sylaiou, S., Liarokapis, F., Grammatiko-poulou, A., Dimitropoulos, K., Grammalidis, N., Partarakis, N., Margetis, G., Drossis, G., Vassiliadi, M., Chalmers, A., Stepha-nidis, C., Thalmann, N. (2018). Mixed Reality Gamified Presence and Storytelling for Virtual Museums. [19] Papagiannakis, G., Lydatakis, N., Kateros, S., Georgiou, S., and Zikas, P., 2018. Transforming Medical Education and Training with VR using M.A.G.E.S. In Proceedings of Siggraph Asia ’18 Posters, Tokyo, Japan, December 04-07, 2018 https://doi.org/10.1145/3283289.3283291 [20] Papagiannakis, G., Zikas, P., Lydatakis, N., Kateros, S., Ken-tros, M., Geronikolakis, E., Kamarianakis, M.N., Kartsonaki, I., Evangelou, G., “MAGES 3.0: Tying the knot of medical VR”. CoRR abs/2005.01180 (2020). [21] Pau Xiberta, Imma Boada, A new e-learning platform for radi-ology education (RadEd), Computer Methods and Programs in Biomedicine, Volume 126, 2016, Pages 63-75, ISSN 0169-2607, 10.1016/j.cmpb.2015.12.022. [22] Pfeiffer-Leßmann, N., Pfeiffer, T. ExProtoVAR: A Lightweight Tool for Experience-Focused Prototyping of Augmented Reality Applications Using Virtual Reality. (2018) 10.1007/978-3-319-92279-9_42. [23] Quiver - 3D Coloring App, https://apps.apple.com/us/app/quiver-3d-coloring-app/id650645305, (last accessed August 31, 2020). [24] Sandor, C., Fuchs, M., Cassinelli, A., Li, H., Newcombe, R., Yamamoto, G., Feiner, S. (2015). Breaking the Barriers to True Augmented Reality. [25] Software Development: Productivity and Context switching, https://medium.com/@mayuminishimoto/software-development-productivity-and-context-switching-66f99b388033 (last accessed September 14, 2020) [26] Stefanidi, E., Arampatzis, D., Leonidis, A., Papagiannakis, G. (2019). BricklAyeR: A Plat-form for Building Rules for AmI Environments in AR. 10.1007/978-3-030-22514-8_39. [27] Unity3D, https://unity.com/, (last accessed September 2, 2020) [28] Unity MARS, https://unity.com/products/unity-mars, (last ac-cessed July 26, 2020). [29] UnityZip, https://github.com/tsubaki/UnityZip, (last accessed September 2, 2020) [30] The XRSI Definitions of Extended Reality (XR), XR Data Clas-sification Framework, XR-DCF Public Working Group XR Safety Initiative, California, USA. [31] Zikas, P., Bachlitzanakis, V., Papaefthymiou, M., Kateros, S., Georgiou, S., Lydatakis, N., Papagiannakis G., Mixed reality se-rious games for smart education. In European Conference on Games Based Learning 2016. ECGBL’16, 2016. [32] Zikas, P., Bachlitzanakis, V., Papaefthymiou, M., Kateros, S., Georgiou, S., Lydatakis, N., Papagiannakis, G., “Mixed Reality Serious Games for smart education”, European Conference on Games Based Learning 2016, ECGBL’16, Paisley, Scotland, October 2016. [33] Zikas, P., Lydatakis, N., Kateros, S., Papagiannakis, G., “Scenior: An Immersive Visual Scripting system of Gamified Training based on VR Software Design Patterns”, 2019, arXiv:1909.05719v1.ings of the 15th Iberoamerican Congress of Digital Graphics, 304-307. SIGraDi. Santa Fe, Argentina, 2011. [6] Geronikolakis, E., Tsioumas, M., Bertrand, S., Loupas, A., Zikas, P., Papagiannakis, G., “New Cross/Augmented Reality Experiences for the Virtual Museums of the Future”. EuroMed (1) 2018: 518-527. [7] Geronikolakis, E., Zikas, P., Kateros, S., Lydatakis, N., Geor-giou, S., Kentros, M., Papagiannakis, G., “A True AR Authoring Tool for Interactive Virtual Museums”. CoRR abs/1909.09429 (2019). [8] Google’s ARCore, https://developers.google.com/ar, (last ac-cessed September 2, 2020) [9] How to port your app to another platform, https://mwdn.com/port-app-another-platform/, (last accessed September 2, 2020) [10] Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G., (Eds).: Mixed Reality and Gamification for Cultural Heritage, Springer, DOI: 10.1007/978-3-319-49607-8, (2017). [11] Ioannides, Marinos, et al., eds. Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection: 7th International Conference, EuroMed 2018, Nicosia, Cyprus, October 29–November 3, 2018, Proceedings. Vol. 11196. Springer, 2018. [12] Jung, Timothy & Tom Dieck, M. Claudia & Lee, Hyunae & Chung, Namho. (2016). Effects of Virtual Reality and Aug-mented Reality on Visitor Experiences in Museum. 10.1007/978-3-319-28231-2_45. [13] Kotis K. ARTIST - a reAl-time low-effoRt mulTi-entity Interac-tion System for creaTing re-usable and optimized MR experiences (2019). Research Ideas and Outcomes 5: e36464. https://doi.org/10.3897/rio.5.e36464. [14] Liarokapis, F., Sylaiou, S., Basu, A., Mourkoussis, N., White, M., Lister, P.F. An Interactive Visualisation Interface for Virtual Museums, Proc. of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, Eurographics As-sociation, Brussels, Belgium, 6-10 Dec, 47-56, 2004. [15] Liarokapis F., Petridis P., Andrews D., de Freitas S. (2017) Multimodal Serious Games Technologies for Cultural Heritage. In: Ioannides M., Magnenat-Thalmann N., Papagiannakis G. (eds) Mixed Reality and Gamification for Cultural Heritage. Springer, Cham. [16] Ott, Michela & Dagnino, Francesca & Pozzi, Francesca & Yil-maz, Erdal & Tsalakanidou, Filareti & Dimitropoulos, Kosmas & Nikos, Grammalidis. (2015). Serious Games to Support Learning of Rare 'Intangible' Cultural Expressions. [17] Papaeftymiou M., Kanakis E.M., Geronikolakis E., Nochos A., Zikas P., Papagiannakis G., “Rapid Reconstruction and Simula-tion of Real Characters in Mixed Reality Environments”, Digital Cultural Heritage Lecture Notes in Computer Science, Vol. 10605, 267-276, 2018. [18] Papagiannakis, G., Geronikolakis, E., Pateraki, M., Bendicho, V.M., Tsioumas, M. Sylaiou, S., Liarokapis, F., Grammatiko-poulou, A., Dimitropoulos, K., Grammalidis, N., Partarakis, N., Margetis, G., Drossis, G., Vassiliadi, M., Chalmers, A., Stepha-nidis, C., Thalmann, N. (2018). Mixed Reality Gamified Presence and Storytelling for Virtual Museums. [19] Papagiannakis, G., Lydatakis, N., Kateros, S., Georgiou, S., and Zikas, P., 2018. Transforming Medical Education and Training with VR using M.A.G.E.S. In Proceedings of Siggraph Asia ’18 Posters, Tokyo, Japan, December 04-07, 2018 https://doi.org/10.1145/3283289.3283291 [20] Papagiannakis, G., Zikas, P., Lydatakis, N., Kateros, S., Ken-tros, M., Geronikolakis, E., Kamarianakis, M.N., Kartsonaki, I., Evangelou, G., “MAGES 3.0: Tying the knot of medical VR”. CoRR abs/2005.01180 (2020). [21] Pau Xiberta, Imma Boada, A new e-learning platform for radi-ology education (RadEd), Computer Methods and Programs in Biomedicine, Volume 126, 2016, Pages 63-75, ISSN 0169-2607, 10.1016/j.cmpb.2015.12.022. [22] Pfeiffer-Leßmann, N., Pfeiffer, T. ExProtoVAR: A Lightweight Tool for Experience-Focused Prototyping of Augmented Reality Applications Using Virtual Reality. (2018) 10.1007/978-3-319-92279-9_42. [23] Quiver - 3D Coloring App, https://apps.apple.com/us/app/quiver-3d-coloring-app/id650645305, (last accessed August 31, 2020). [24] Sandor, C., Fuchs, M., Cassinelli, A., Li, H., Newcombe, R., Yamamoto, G., Feiner, S. (2015). Breaking the Barriers to True Augmented Reality. [25] Software Development: Productivity and Context switching, https://medium.com/@mayuminishimoto/software-development-productivity-and-context-switching-66f99b388033 (last accessed September 14, 2020) [26] Stefanidi, E., Arampatzis, D., Leonidis, A., Papagiannakis, G. (2019). BricklAyeR: A Plat-form for Building Rules for AmI Environments in AR. 10.1007/978-3-030-22514-8_39. [27] Unity3D, https://unity.com/, (last accessed September 2, 2020) [28] Unity MARS, https://unity.com/products/unity-mars, (last ac-cessed July 26, 2020). [29] UnityZip, https://github.com/tsubaki/UnityZip, (last accessed September 2, 2020) [30] The XRSI Definitions of Extended Reality (XR), XR Data Clas-sification Framework, XR-DCF Public Working Group XR Safety Initiative, California, USA. [31] Zikas, P., Bachlitzanakis, V., Papaefthymiou, M., Kateros, S., Georgiou, S., Lydatakis, N., Papagiannakis G., Mixed reality se-rious games for smart education. In European Conference on Games Based Learning 2016. ECGBL’16, 2016. [32] Zikas, P., Bachlitzanakis, V., Papaefthymiou, M., Kateros, S., Georgiou, S., Lydatakis, N., Papagiannakis, G., “Mixed Reality Serious Games for smart education”, European Conference on Games Based Learning 2016, ECGBL’16, Paisley, Scotland, October 2016. [33] Zikas, P., Lydatakis, N., Kateros, S., Papagiannakis, G., “Scenior: An Immersive Visual Scripting system of Gamified Training based on VR Software Design Patterns”, 2019, arXiv:1909.05719v1.