Defining Preferred and Natural Robot Motions in Immersive Telepresence from a First-Person Perspective
Katherine J. Mimnaugh, Markku Suomalainen, Israel Becerra, Eliezer Lozano, Rafael Murrieta-Cid, Steven M. LaValle
DDefining Preferred and Natural Robot Motions in ImmersiveTelepresence from a First-Person Perspective
Katherine J. Mimnaugh [email protected] of OuluOulu, Finland
Markku Suomalainen
University of OuluOulu, [email protected]
Israel Becerra
Centro de Investigación enMatemáticasGuanajuato, México
Eliezer Lozano
Centro de Investigación enMatemáticasGuanajuato, México
Rafael Murrieta-Cid
Centro de Investigación enMatemáticasGuanajuato, México
Steven M. LaValle
University of OuluOulu, Finland
Figure 1: An overview of immersive robotic telepresence through an HMD; the remote user can be telepresent in a remotelocation and feel as if they are really there.
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected].
VAM-HRI ’21, March 08–11, 2021, Virtual Conference © 2021 Association for Computing Machinery.ACM ISBN 978-x-xxxx-xxxx-x/YY/MM...$15.00
ABSTRACT
This paper presents some early work and future plans regardinghow the autonomous motions of a telepresence robot affect a per-son embodied in the robot through a head-mounted display. Weconsider the preferences, comfort, and the perceived naturalness ofaspects of piecewise linear paths compared to the same aspects on asmooth path. In a user study, thirty-six subjects (eighteen females)watched panoramic videos of three different paths through a simu-lated museum in virtual reality and responded to questionnairesregarding each path. We found that comfort had a strong effecton path preference, and that the subjective feeling of naturalness a r X i v : . [ c s . R O ] F e b AM-HRI ’21, March 08–11, 2021, Virtual Conference Mimnaugh, et al. also had a strong effect on path preference, even though peopleconsider different things as natural. We describe a categorizationof the responses regarding the naturalness of the robot’s motionand provide a recommendation on how this can be applied morebroadly. Although immersive robotic telepresence is increasinglybeing used for remote education, clinical care, and to assist peo-ple with disabilities or mobility complications, the full potentialof this technology is limited by issues related to user experience.Our work addresses these shortcomings and will enable the futurepersonalization of telepresence experiences for the improvement ofoverall remote communication and the enhancement of the feelingof presence in a remote location.
CCS CONCEPTS • Human-centered computing → Virtual reality ; User stud-ies ; Empirical studies in HCI ; •
Computer systems organiza-tion → Robotic autonomy ; Robotic control ; KEYWORDS
Human-Robot Interaction, Adaptive Robotic Behavior, Telepres-ence, Virtual Reality, Robotics, Motion Planning, User Studies
ACM Reference Format:
Katherine J. Mimnaugh, Markku Suomalainen, Israel Becerra, Eliezer Lozano,Rafael Murrieta-Cid, and Steven M. LaValle. 2021. Defining Preferred andNatural Robot Motions in Immersive Telepresence from a First-Person Per-spective. In
Proceedings of VAM-HRI ’21: ACM/IEEE International Workshopon Virtual, Augmented, and Mixed-Reality for Human-Robot Interactions(VAM-HRI ’21).
ACM, New York, NY, USA, 4 pages.
Currently, the most scalable technology with the potential to achievethe feeling of being immersed in a remote location is a panoramiccamera streaming to a Virtual Reality (VR) Head-Mounted Dis-play (HMD); see Fig. 1. The immersion provided by an HMD canfacilitate more natural interaction and thus ease the difference be-tween telepresence and physical presence [10]. Unfortunately, thereare several concerns regarding the use of telepresence with an HMDthat do not typically occur in telepresence with a Fixed Naked-eyeDisplay (FND, a traditional display such as on a computer or aphone). The velocities that the robot uses or the passing distancesto objects and walls, which would otherwise be comfortable inFND telepresence, can be uncomfortable in immersive telepresence.Second, a serious consequence of using an HMD is
VR sickness [6],which can result in the user experiencing nausea and vertigo.Although there are known methods to reduce VR sickness [2],such as reducing the Field-of-View (FOV) [11], these techniquesmay have a negative effect on the feeling of being present [13].However, by planning autonomous trajectories for the robot whileavoiding motions known for contributing to VR sickness, the feel-ing of sickness can be decreased without sacrificing the feeling ofpresence. Additionally, speeds and distances to objects can alsobe taken into account to make the experience comfortable for theuser. Thus, to effectively adapt the robot’s motion planning criteriapriorities to each user, the impacts of these components on the usermust be analyzed. For example, turns have been shown to play arole in VR sickness, [3], but there is evidence that the duration of
Figure 2: A screenshot inside the virtual museum, seen byparticipants from the point of view of the virtual telepres-ence robot. The entrance hallways is seen on top, and themain gallery in the museum is shown on the bottom. the turn is more important than the total angle of rotation [14]. Thisknowledge was used to plan piecewise linear trajectories for a robot,with rotations in place, which resulted in reduced VR sickness inan immersive telepresence experience [1].However, there are other aspects to consider besides reducingVR sickness; issues such as the path not feeling natural or qualitiesof the turns when rotating in place may detract from the experience.These issues are typically not a problem in telepresence through anFND, since it is the immersion experienced through the near-eyedisplays of an HMD that allow the user to really feel them. Theseissues are also not frequently studied in pure VR research, sincemotions in VR are usually either directly controlled by the user, orteleportation is used. Although these methods for locomotion in VRmay be more comfortable for the user, teleportation is infeasible intelepresence and manual control is perceived as bothersome in largeenvironments [9]. Thus, there is a need to research autonomousmotions to overcome these obstacles in immersive telepresence.Negative impacts, like VR sickness, are not the only concernwhen designing autonomous telepresence robot trajectories. Aninteresting, yet unexplored, positive element of the telepresenceexperience is the naturalness of the robot’s motions. Some workhas discussed attempts to make robot motion more natural [5] ordeployed voiced commands to make robots perform more naturally[8]. However, to the authors’ knowledge, there is no widely accepteddefinition of natural motions for mobile robots. Moreover, from theperspective of the user embodying the telepresence robot, theremay not currently be any research on whether people even prefermotions that they would perceive as natural, or more generallywhat kind of trajectories are preferred for an autonomously movingtelepresence robot to take. efining Preferred and Natural Robot Motions in Immersive Telepresence from a First-Person Perspective VAM-HRI ’21, March 08–11, 2021, Virtual Conference
Figure 3: Conceptualization of the dimensions on which sub-jects consider movement to be natural. Horizontally is thetime frame dimension. Things closer to the origin are moreimmediate (turns), whereas away from the origin, longerterm, relates to things like the path. On the generality di-mension, things closer to the origin are specific to a place(in this museum) whereas things away from the origin aremore general (in any setting). The units of each dimensionare not discrete and can vary by individual.
We performed an experiment where the subjects wore an HMD andwatched three pre-recorded trajectories of a virtual telepresencerobot moving through a virtual museum. A screenshot from insidethe museum can be seen in Fig. 2, and the videos (in 2D format)can be viewed at: https://youtu.be/FPbrIlw3OhY. The data wascollected in February 2020, before coronavirus restrictions wereenacted in Finland. Two of the trajectories were piecewise lineartrajectories with 45 ◦ or 90 ◦ turns, optimized for the least amountof turns and for the shortest path (see [1] for the path planning),while the third path was a smooth path created with a Rapidly-exploring Random Trees (RRT) algorithm [7]. We selected the RRTbecause it is commonly used in robot motion planning, it respectsthe differential drive robot system’s dynamics while planning thetrajectory in 5-dimensional space, and because there is evidencethat RRT paths are considered to be human-like [12]. We askedthe subjects to rate aspects of the paths on a Likert-scale, and toselect their preferred path, the most comfortable path, and the mostnatural path. We also asked them open-ended questions regardingwhy they made those selections.The environments were created with the Unity 3D game engineand displayed to subjects as 360 degree videos played throughthe Virtual Desktop application into an Oculus Rift S. Subjectswere seated in a stationary chair and able to look around whilethe videos were played. Thirty-six subjects from campus and thelocal community were run in a gender and path counterbalancedwithin-subjects design, with each subject viewing all three of thestimulus videos and completing a Simulator Sickness Questionnaire(SSQ, [4]) and questionnaires regarding their experience on each path. Subjects signed a consent form before participating and weredebriefed after the study was completed. We set out to examine the interplay between the preference, com-fort, and perceived naturalness of robot motions for first-personexperiences in VR telepresence. We found the most salient aspectregarding comfort was in relation to the turns. Comfort had a strongassociation with preference, but not with naturalness. We also foundthat, after comfort, preference for a particular path was influencedthe most by forward speed and the turns. Preference was stronglyassociated with the users’ perceived naturalness, which was pri-marily determined by the ability to see salient objects, the distanceto the walls and objects, as well as the turns. Participants favoredthe paths that had a one meter per second forward speed and ratedthe path with the least amount of turns as the most comfortable.The piecewise linear paths with rotation in place were preferredand selected as the most natural more often than the smooth pathwhere rotation and translation occurred simultaneously. Finally,we also found that the robot’s speed and passing distances had asignificant impact on the users, and must be carefully consideredwhen implementing immersive telepresence.Naturalness is a topic often mentioned as a goal in robotics pub-lications, but there is no widely accepted definition of naturalnessand it can mean different things to different people. In order tofurther elucidate how naturalness is thought of by people inter-acting with a robot, we asked participants open-ended questionsregarding the naturalness of the robot’s motions. We examined theanswers on why people considered a certain path to be the mostnatural and found that there seem to be two dimensions (see Fig. 3)on which people interpreted the given definition of naturalness,"how you would have moved through the museum if you werein control": time frame, and generality. On the time frame dimen-sion, responses regarding either how the robot performed the turns(fifteen comments, twenty-five percent of the total comments onnaturalness) or how close to walls and objects the robot moved(thirteen comments, twenty-two percent) can be considered to re-flect something more immediate, while comments related to thingsfurther in time referred to aspects of the whole path through themuseum (fifteen comments, twenty-five percent). Regarding thegenerality dimension that describes either context-specific or moregeneral behavior, subjects specifically mentioned the museum set-ting in seven answers spread across all paths, but some commentsregarding turning, for example, could be considered as not specificto the context.The two dimensions in Fig. 3 depict a categorization of certainaspects of the answers regarding naturalness when subjects werequeried on how they would have moved through the museum ifthey were in control. Answers about naturalness seemed to gen-erally reflect these two dimensions, though how naturalness isconceptualized will include significant individual variation. Fur-thermore, it should be noted that these answers are a expressionof what participants though of as natural in a particular context,the virtual museum. Whereas some answers were specific to thissetting ( "I would visit one side first and then go to the other side of themuseum, slowly exploring. So the first one was more natural" ), others
AM-HRI ’21, March 08–11, 2021, Virtual Conference Mimnaugh, et al. discussed naturalness more generally ( "the walking speed was closet[sic] to my own" ). Although the situation in which these answerswere generated was quite distinct, the authors believe that peoplethink of naturalness on these two dimensions in many other casesand situations, which our subsequent work will examine.
When attempting to make a robot move "naturally," researchersshould take individual variation into account and consider thetwo dimensions on which it might be perceived. For example, re-searchers can consider that natural motion in a certain contextmight not be natural in other contexts, which we found in answerswhere subjects contrasted how they would move in a museum ( "Ithink in real life I would move similarly to the 3rd path, because itwould allow me to view the items better (closer). Also, it simulatesthe hesitation of a visitor when going in around a museum" ) versushow they consider themselves to move more generally ( "Walkingthrough a corridor in straight line felt more natural than it was in thefirst path where the movement was from one corner towards another" ).Researchers can also consider that the naturalness of parts of thepath like the speed and how the turns are executed ( "This speed ismore natural for a scenario or location such as this. However, the turnswere a bit sharp and you feel like you’re too close to the objects" ),or the whole path that is taken ( "It felt most like a path that anactual human would take, getting closer to inspect the paintings" ),can each be considered to exhibit naturalness or a lack thereof. Allof the individual pieces of a trajectory may be construed as naturalor not natural in conflicting ways, so there does not seem to beone universal method to categorize robot motion as natural or not.Therefore, care should be exercised in the third-party classificationof the naturalness of robot motion; indeed what a roboticist de-signing the motion or a reviewer reading the paper might assessas natural may be quite distinct from what a general member ofthe public may appraise as being natural. Thus the best determina-tion of naturalness is, therefore, attained through the evaluation ofresearch participants.
Future work will will test these same trajectories through the mu-seum, except without any artwork present. Since many peopleregarded path qualities to be related to their ability to see objects inthe museum, the artwork had a significant impact on the subjects’responses regarding preference, comfort, and naturalness. In con-trast to the first study with the art, we hypothesize that the turnswill still have the largest impact on comfort, and that the path withthe least amount of turns will be the preferred path when movingthrough an empty room. We also postulate that the least turns pathwill be rated as the most natural because it is piecewise linear. Theresulting comparisons will further elucidate how naturalness isconceptualized more broadly and will provide further clarificationon user preferences from a first person perspective regarding turnspeed, forward speed, trajectories, and comfort for the physicalmovements of immersive telepresence robots.
REFERENCES [1] I. Becerra, M. Suomalainen, E. Lozano, K. J. Mimnaugh, R. Murrieta-Cid, and S. M.LaValle. 2020. Human Perception-Optimized Planning for Comfortable VR-BasedTelepresence.
IEEE Robotics and Automation Letters
5, 4 (2020), 6489–6496.[2] E. Chang, H. T. Kim, and B. Yoo. 2020. Virtual Reality Sickness: A Review ofCauses and Measurements.
International Journal of Human-Computer Interaction
36, 17 (2020), 1–25.[3] Senqi Hu, Kathleen A McChesney, Kathryn A Player, Amy M Bahl, Jessica BBuchanan, and Jason E Scozzafava. 1999. Systematic investigation of physiologicalcorrelates of motion sickness induced by viewing an optokinetic rotating drum.
Aviation, space, and environmental medicine (1999).[4] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal. 1993. Simulatorsickness questionnaire: An enhanced method for quantifying simulator sickness.
The international journal of aviation psychology
3, 3 (1993), 203–220.[5] Junghyun Kwon and Frank C Park. 2008. Natural movement generation usinghidden markov models and principal components.
IEEE Transactions on Systems,Man, and Cybernetics, Part B (Cybernetics)
38, 5 (2008), 1184–1194.[6] S. M. LaValle. 2021.
Virtual reality . Cambridge University Press.[7] S. M. LaValle and J. J. Kuffner Jr. 2001. Randomized kinodynamic planning.
Theinternational journal of robotics research
20, 5 (2001), 378–400.[8] Amy LaViers. 2019. Make robot motions natural.
Nature
565 (2019), 422–424.[9] Irene Rae and Carman Neustaedter. 2017. Robotic telepresence at scale. In
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems .313–324.[10] Mel Slater and Sylvia Wilbur. 1997. A framework for immersive virtual envi-ronments (FIVE): Speculations on the role of presence in virtual environments.
Presence: Teleoperators & Virtual Environments
6, 6 (1997), 603–616.[11] Joel Teixeira and Stephen Palmisano. 2020. Effects of dynamic field-of-viewrestriction on cybersickness and presence in HMD-based virtual reality.
VirtualReality (2020), 1–13.[12] Annemarie Turnwald and Dirk Wollherr. 2019. Human-like motion planningbased on game theoretic decision making.
International Journal of Social Robotics
11, 1 (2019), 151–170.[13] S. Weech, S. Kenny, and M. Barnett-Cowan. 2019. Presence and cybersickness invirtual reality are negatively related: a review.
Frontiers in psychology
10 (2019),158.[14] C. Widdowson, I. Becerra, C. Merrill, R. F. Wang, and S. LaValle. 2019. AssessingPostural Instability and Cybersickness Through Linear and Angular Displace-ment.