Discussing the Risks of Adaptive Virtual Environments for User Autonomy
DDiscussing the Risks of Adaptive VirtualEnvironments for User Autonomy
Tobias Drey
Institute of Media InformaticsUlm University, Ulm, [email protected]
Enrico Rukzio
Institute of Media InformaticsUlm University, Ulm, [email protected]
Presented at the workshop “Exploring Potentially Abusive Ethical, Social and PoliticalImplications of Mixed Reality Research in HCI”, CHI ’20, April 25-30, Honolulu, HI,USA.Copyright held by author(s).
Abstract
Adaptive virtual environments are an opportunity to supportusers and increase their flow, presence, immersion, andoverall experience. Possible fields of application areadaptive individual education, gameplay adjustment,professional work, and personalized content. But whobenefits more from this adaptivity, the users who can enjoya greater user experience or the companies or governmentswho are completely in control of the provided content. Whilethe user autonomy decreases for individuals, the power ofinstitutions raises, and the risk exists that personal opinionsare precisely controlled. In this position paper, we will arguethat researchers should not only propose the benefits oftheir work but also critically discuss what are possibleabusive use cases. Therefore, we will examine two usecases in the fields of professional work and personalizedcontent and show possible abusive use.
Author Keywords serious games; educational games; virtual reality;adaptivity; player state assessment; stealth assessment;user autonomy; privacy;
CCS Concepts • Security and privacy → Privacy protections; • Human-centered computing → Virtual reality; • Appliedcomputing → Interactive learning environments; a r X i v : . [ c s . H C ] J a n ntroduction Adaptivity provides an opportunity to provide individual userexperience (UX) and increase flow, presence, andimmersion. Related work has shown, that behavioralindicators and biometric data for hand [6, 9] and footmovements [5, 8], as well as gaze behavior [7, 11, 12], canbe used to adapt virtual environments and virtual reality(VR) [2, 4]. The use of VR head-mounted displays (HMDs)provides the possibility to increase these effects byencapsulating users to their own environment.The intention behind adaptivity is that it is suitable toincrease the effectiveness of education and professionalwork, the overall gameplay experience of video games, andto introduce perfectly matching personal content like a newsfeed. But the risk exists that adaptivity and all its benefitscome at the cost of losing freedom and control. Whenever amachine makes a decision and adapts a system for us toour possible favor, then it also withholds information orpossible solutions from us. On the positive side, this can bebeneficial as it helps us to filter a lot of unnecessaryinformation so that we have more time to concentrate onimportant things. On the negative side, it provides thepossibility to hide information and install unnoticedcensorship. This misuse is often ignored when newadaptive systems are developed and evaluated asresearchers are excited about what they did, but a profounddiscussion in a paper should also consider negative andpossible abusive aspects.In this position paper, we discuss two use cases and showhow good and helpful approaches can be abused. We willconsider a professional work VR application and adaptivepersonalized content based on user profiles.
Discussion of Abusive User Monitoring
Problem Definition
Adaptive educational games help to provide individuallearning experiences by constantly monitoring the user [13].This is beneficial for the learning context. Techniques weredeveloped to assess different capabilities of the user likespatial abilities [10] or problem understanding [3]. Most ofthese techniques have in common that they are completelyunrecognized by the user. Therefore, the risk exists that auser is monitored and assessed without knowledge.
Adaptive Individual Work Environment
One possible example is an employee working withcomputer-aided design (CAD) VR software. Similar, asproposed by Drey et al. [4] and Bye et al. [2], this softwaremonitors all body movements, including hands, feet, head,and even the gaze behavior. Furthermore, the workflow ismonitored too, and the personal capabilities were assessedbased on measurements like spatial abilities. This couldthen be automatically reported to the management whichresults in a possible promotion or a job downgrade. Besidesreporting to the management, the software could select thetasks for the employee based on the assessed skills. Thiscould be beneficial for the company as employees alwayswork on tasks perfectly fitting to their skills, but it is alsoproblematic as they have no possibility to improve their skillswith challenging tasks. When this adaptive preselection isdone completely non-transparent, then this is a sort ofcensorship.
Adaptive Personalized Content Based on User Profiles
All major IT companies like Facebook, Google, Apple, orMicrosoft create user profiles based on user activity [1].These profiles can be enriched with data gained throughadaptive games with leisure or educational content. Themethods proposed by Drey et al. [4] and Bye et al. [2]rovide new possibilities to create a much more complexuser profile than today. They tell a lot about how usersunderstand and interact with their environment. This data isused, at the moment, mostly for advertisement and contentpredictions. Personally perceived interesting content is apositive thing, but it creates on the other side an individualbubble, which leads to a sort of automatic censorship.The more data is collected, it gets more likely that there isonly personalized content and no "uncensored" generalone. In a dystopic future, companies and governments useall data they are gathering from their users or citizens tomonitor and assess them. The techniques now developedfor positive adaptive environments are used then to createcensored environments. Users cannot opt-out from this anddo not know how they are monitored due to the use ofunobtrusive techniques. The public opinion is controlled aseveryone gets only censored and personally fittedinformation.The reason why no general and uncontrolled way to getinformation exists anymore is that newspapers and otherindependent organizations got bankrupt because userspreferred their personalized information. As censorshipincreased slowly during the time, they were not aware ofhow they were manipulated. This was a slow, unobtrusive,but constant process until the power of the companies andgovernments was high enough to control public opinion.
Conclusion
The described two scenarios in the fields of a professionalworking environment and personalized content are clearlydystopic worst-case scenarios. There are laws andregulations who should protect employees and citizens frompermanent monitoring. Nevertheless, it is important thateveryone is aware of what is technically possible to control whether laws and regulations are kept. Therefore, we arguethat researchers should consider possible abusive usecases as well and point out this to the readers of theirpapers. Whenever this is not done sufficiently by theauthors themselves, this should be done by the communityby raising concerns.At the workshop, I would like to talk about these dystopicscenarios and discuss who of the participants share myconcerns about adaptive virtual environments.Furthermore, I would like to discuss possible actions toprevent abusive use.
Acknowledgments
This work was conducted within the project AuCity 2, fundedby the Federal Ministry of Education and Research (BMBF).
REFERENCES [1] 2020. Are you ready? This is all the data Facebookand Google have on you | Dylan Curran | Opinion | TheGuardian. (Feb 2020). (Accessed on 02/20/2020).[2] Kent Bye, Diane Hosfelt, Sam Chase, Matt Miesnieks,and Taylor Beck. 2019. The Ethical and PrivacyImplications of Mixed Reality. In
ACM SIGGRAPH2019 Panels (SIGGRAPH ’19) . Association forComputing Machinery, New York, NY, USA, ArticleArticle 4, 2 pages.
DOI: http://dx.doi.org/10.1145/3306212.3328138 [3] Samantha Jane Clarke, Daryl J. Peel, Sylvester Arnab,Luca Morini, Helen Keegan, and Oliver Wood. 2017.EscapED: A Framework for Creating EducationalEscape Rooms and Interactive Games to ForHigher/Further Education.
International Journal oferious Games
4, 3 (Sep. 2017).
DOI: http://dx.doi.org/10.17083/ijsg.v4i3.180 [4] Tobias Drey, Pascal Jansen, Fabian Fischbach, JulianFrommel, and Enrico Rukzio. 2020. Towards ProgressAssessment for Adaptive Hints in Educational VirtualReality Games. In
Extended Abstracts of the 2020 CHIConference on Human Factors in Computing Systems(CHI ’20) . Association for Computing Machinery, NewYork, NY, USA, Article Paper LBW324, 9 pages.
DOI: http://dx.doi.org/10.1145/3334480.3382789 [5] Yasmin Felberbaum and Joel Lanir. 2018. BetterUnderstanding of Foot Gestures: An Elicitation Study.In
Proceedings of the 2018 CHI Conference on HumanFactors in Computing Systems . ACM, 334.[6] Donald Glowinski, Nele Dael, Antonio Camurri,Gualtiero Volpe, Marcello Mortillaro, and KlausScherer. 2011. Toward a minimal representation ofaffective gestures.
IEEE Transactions on AffectiveComputing
2, 2 (2011), 106–118.[7] Michael Lankes and Andreas Haslinger. 2019. Lost &Found: Gaze-based Player Guidance Feedback inExploration Games. In
Extended Abstracts of theAnnual Symposium on Computer-Human Interaction inPlay Companion Extended Abstracts . ACM, 483–492.[8] Ye Pan and Anthony Steed. 2019. How Foot TrackingMatters: The Impact of an Animated Self-Avatar onInteraction, Embodiment and Presence in SharedVirtual Environments.
Frontiers in Robotics and AI arXiv preprint arXiv:1402.5047 (2014).[10] Roger N. Shepard and Jacqueline Metzler. 1971.Mental Rotation of Three-Dimensional Objects.
Science
DOI: http://dx.doi.org/10.1126/science.171.3972.701 [11] Jose L Soler-Dominguez, Jorge D Camba, ManuelContero, and Mariano Alcañiz. 2017. A proposal for theselection of eye-tracking metrics for the implementationof adaptive gameplay in virtual reality based games. In
International Conference on Virtual, Augmented andMixed Reality . Springer, 369–380.[12] Veronica Sundstedt, Matthias Bernhard, EfstathiosStavrakis, Erik Reinhard, and Michael Wimmer. 2013.
Visual Attention and Gaze Behavior in Games: AnObject-Based Approach . Springer London, London,543–583.
DOI: http://dx.doi.org/10.1007/978-1-4471-4769-5_25 [13] Stefanie Vanbecelaere, Katrien Van den Berghe,Frederik Cornillie, Delphine Sasanguie, Bert Reynvoet,and Fien Depaepe. 2019. The effectiveness ofadaptive versus non-adaptive learning with digitaleducational games.
Journal of Computer AssistedLearning (2019).