Enhancing autonomy transparency: an option-centric rationale approach
EEnhancing autonomy transparency: an option-centric rationale approach
Ruikun LuoRobotics Institute, University of Michigan, Ann ArborNa DuIndustrial and Operations Engineering, University of Michigan, Ann ArborX. Jessie YangIndustrial and Operations Engineering, University of Michigan, Ann Arbor
Manuscript type:
Original Research
Running head:
Enhancing autonomy transparency
Word count:
Corresponding author:
X. Jessie Yang, 1205 Beal Avenue, Ann Arbor, MI48109,Email: [email protected]
Acknowledgement:
We would like to thank Kevin Y. Huang for his assistance in datacollection. a r X i v : . [ c s . H C ] A ug bstract While the advances in artificial intelligence and machine learning empower a newgeneration of autonomous systems for assisting human performance, one major concernarises from the human factors perspective: Humans have difficulty decipheringautonomy-generated solutions and increasingly perceive autonomy as a mysterious blackbox. The lack of transparency contributes to the lack of trust in autonomy andsub-optimal team performance. To enhance autonomy transparency, this studyproposed an option-centric rationale display and evaluated its effectiveness. Wedeveloped a game
Treasure Hunter wherein a human uncovers a map for treasures withthe help from an intelligent assistant, and conducted a human-in-the-loop experimentwith 34 participants. Results indicated that by conveying the intelligent assistant’sdecision-making rationale via the option-centric rationale display, participants hadhigher trust in the system and calibrated their trust faster. Additionally, higher trustled to higher acceptance of recommendations from the intelligent assistant, and in turnhigher task performance.
Keyword:
Transparent autonomy, Transparent automation, Design rationale,Trust in automation, Trust calibration, Propositional logic. 2 . INTRODUCTION
While the advances in artificial intelligence and machine learning empowers a newgeneration of autonomous systems for assisting human performance, one major concernarises from the human factors perspective: Human agents have difficulty decipheringautonomy-generated solutions and increasingly perceived autonomy as a mysteriousblack box. The lack of transparency contributes to the lack of trust in autonomy andsub-optimal team performance (Chen & Barnes, 2014; de Visser, Pak, & Shaw, 2018;Du et al., 2019; Endsley, 2017; Lyons & Havig, 2014; Lyons et al., 2016; Yang,Unhelkar, Li, & Shah, 2017).There are multiple definitions of autonomy transparency, to name a few: “the[degree of] shared intent and shared awareness between a human and a machine (Lyons& Havig, 2014)”, “the extent to which an autonomous agent can convey its intent,performance, future plans and reasoning process (Chen et al., 2014)”, “a mechanism toexpose the decision-making of a robot (Theodorou, Wortham, & Bryson, 2017)”, “theunderstandability and predictability of their actions (Endsley, 2017)”, “the ability forthe automation to be inspectable or viewable in the sense that its mechanisms andrationale can be readily known (Miller, 2018)”. Despite the lack of a universaldefinition, a fairly consistent pattern can be observed: a transparent autonomy shouldcommunicate to the human agent the autonomy’s ability and performance, itsdecision-making logic and rationale, and its intent and future plans.Although autonomy transparency was only recently defined, research has beenconducted to convey certain aspects of autonomy-generated solutions. One body ofhuman factors research has concentrated on conveying likelihood information in theform of automation reliability, (un)certainty, and confidence. Some studies revealed thatlikelihood information significantly helped human operators calibrate their trust andenhance human-automation team performance (McGuirl & Sarter, 2006; Walliser, deVisser, & Shaw, 2016; Wang, Jamieson, & Hollands, 2009). Other studies reported thathuman operators did not trust or depend on automated decision aids appropriately evenwhen the likelihood information was disclosed (Bagheri & Jamieson, 2004; Fletcher, 3artlett, Cockshell, & McCarley, 2017). Recently, Du, Huang, and Yang (in press)proposed a framework for reconcile the mixed results and showed that not all likelihoodinformation is equal in aiding human-autonomy team performance. Presenting thepredictive values and the overall success likelihood is more beneficial than presentingthe hit and correct rejection rates.Another body of research has investigated the impact of providing hand-craftedexplanations of autonomy’s behaviors. For example, Dzindolet, Peterson, Pomranky,Pierce, and Beck (2003) showed that providing hand-drafted explanations ofautomation failures can lead to increased trust. The studies of Koo et al. (2014) andKoo, Shin, Steinert, and Leifer (2016) showed that by informing the drivers of thereasons for automated breaking (e.g. road hazard ahead) decreased drivers’ anxiety,increased their sense of control, preference and acceptance. Similarly, Du et al. (2019)found that speech output explaining why and how the automated vehicle is going totake certain actions was rated higher on trust, preference, usability and acceptance.More recently, research has gone beyond either conveying likelihood informationor relying on hand-crafted explanation, and has formally defined and examinedautonomy transparency. Notably, Mercado et al. (2016) proposed the situationawareness (SA)-based agent transparency model to convey information supporting thehuman agent’s perception, comprehension, and projection of an intelligent assistant’srecommendations.In this study, we wished to propose the option-centric rationale approach forenhancing autonomy transparency. Inspired by the research on design rationale, theoption-centric rationale approach explicitly displays the option space (i.e. all thepossible options/actions that an autonomy could take) and the rationale why aparticular option is the most appropriate at a given context. Design rationale is an areaof design science focusing on the “representation for explicitly documenting thereasoning and argumentation that make sense of a specific artifact (MacLean, Young,Bellotti, & Moran, 1991)”. Its primary goal is to support designers and other stakeholders by recording the argumentation and reasoning behind the design process. The 4heoretical underpinning for design rationale is that for designers what is important isnot just the specific artifact itself but its other possibilities – why an artifact is designedin a particular way compared to how it might otherwise be.
Figure 1 . An illustration of the QOC notation. Adapted from MacLean et al. (1991).One major approach to representing design rationale is design space analysis. Ituses a semi-formal notation called QOC (Questions, Options, and Criteria):
Questions identify key design issues,
Options provide possible answers to the Questions, and
Criteria illustrate the criteria used for assessing and comparing the Options. The QOCnotation creates an explicit representation of a structured space of design alternativesand the consideration for choosing among the different choices. It has been widelyapplied in the design of human-machine interface (MacLean et al., 1991; Oulasvirta,Feit, Lähteenlahti, & Karrenbauer, 2017; Sutcliffe et al., 2018). Figure 1 illustrates theQOC representation of the design space of the scroll bar mechanism in the XeroxCommon Lisp (XCL) environment. The design question of interest is how to display thescroll bar. There are two options: the scroll bar appears permanently or only appearswhen the cursor is moved over the edge of a window. Choosing among various optionsrequires a range of considerations and reasoning over those considerations. For the scrollbar example, the reasoning criteria includes low user effort, screen compactness andcontinuous feedback to the user. The QOC notation also provides a way to represent an
Assessment of whether an option does or does not satisfy a criterion. The “permanent”5ption ensures continuous feedback and low user effort at the cost of screen space.
2. THE PRESENT STUDY
In the present study, we proposed the option-centric rationale display forenhancing autonomy transparency, and evaluated its effectiveness via ahuman-in-the-loop experiment. In the experiment, a human operator uncovered a mapfor treasures with the help from an intelligent assistant. The intelligent assistant’sdecision-making rationale are conveyed in the option-centric rationale display. Wetested the following hypotheses:First, the option-centric rationale display explicitly explores the option space (i.e.all the possible options/actions that an autonomy could take on) and present therationale why a particular option is the most appropriate at a given context. Weexpected that the enhanced transparency will lead to higher trust: H1 : When the option-centric rationale display is present, human agents will havehigher trust in the autonomous agent.Second, trust has been recognized as one, if not the most, crucial factordetermining the use of automated or autonomous technologies (Hoff & Bashir, 2015).Trust in automation is defined as the "attitude that an agent will help achieve anindividual’s goals in situations characterized by uncertainty and vulnerability (Lee &See, 2004)". The underlying idea is that a human agent’s decision whether or not to usean automated or autonomous technology depends on the extent to which he or shetrusts it. When trust is high, the human agent is more likely to use the technology.Besides the research focus on trust, several researchers have proposed that the humanagent’s dependence behavior is inversely related to his or her self-confidence inperforming the task by themselves manually without the assistance of the intelligentagent (de Vries, Midden, & Bouwhuis, 2003; Kantowitz, Hanowski, & Kantowitz, 1997;Moray, Inagaki, & Itoh, 2000). Therefore, we hypothesized that: H2 : Trust and self-confidence affect people’s acceptance behaviors. Higher trust willlead to higher acceptance and higher self-confidence will reduce acceptance. 6hird, Parasuraman and Riley (1997) categorized different types of automationuse and showed disusing a reliable automation harmed task performance. In the presentstudy, the intelligent assistant is a knowledge-based agent and reasons usingpropositional logic. Therefore, we hypothesized that: H3 : Given a highly capable autonomous agent, higher acceptance leads to higherperformance.Last, most existing studies on trust in automation, or more recently, trust inautonomy, measured trust once at the end of an experiment via a questionnaire (a"snapshot" of trust). Only a limited number of studies have viewed trust as a dynamicvariable that can strength or decay over time (Lee & Moray, 1992; Manzey,Reichenbach, & Onnasch, 2012; Yang et al., 2017; Yang, Wickens, & Hölttä-Otto, 2016).Prior studies showed that human agents calibrate their trust based on their experienceinteracting with the automated or autonomous technology (Yang et al., 2017). Withenhanced autonomy transparency, we expected to see a faster trust calibration process: H4 : People adjust their trust in the autonomous agent as they gain more experienceworking with the autonomous agent. In particular, with the option-centric rationale display, the rate of adjustment will be faster.
3. METHOD
This research complied with the American Psychological Association code of ethicsand was approved by the Institutional Review Board at the University of Michigan.
Thirty-four participants (Age: Mean = 21.17 years, SD = 1.66 years) took part inthe experiment. All participants had normal or corrected-to-normal sight and hearing.Participants were compensated with $5 upon completion of the experiment. In addition,there was a chance to obtain an additional bonus of 1 to 20 dollars based on theirperformance. 7 .2 Simulation testbed We developed an experimental testbed –
Treasure Hunter , adapted from theWumpus world game (Russell & Norvig, 2010). In the game, the participant acts as ahunter to find the gold bar in the map with the help of an intelligent assistant(Figures 2a & 2b). Each step, the hunter can move to an unvisited location which isconnected to the visited locations. Figure 2b shows that the hunter moves from A1 toA2 and then to B1. On the way to the treasure, the hunter might fall into a pit (shownin C1 in Figure 2a) or encounter a wumpus (shown in B3 in Figure 2a). The huntergathers information about his or her surroundings by a set of sensors. The sensors willreport a stench when the wumpus is in an adjacent location (shown as B2, A3, C3, B4in Figure 2a) and a breeze when a pit is in an adjacent location (shown as B1, C2, D1in Figure 2a). There is one and only one gold bar/wumpus in a map. However, theremight be one or multiple pits in a map. Each element - a pit, a wumpus, or a gold bar -occupies a unique location on the map. (a) (b)
Figure 2 . (a) An example map in Treasure Hunter. Each square is denoted by the rownumber (from 1 to 4) and the column number (from A to D). (b) First two steps of ahunter moving in the map.Table 1 shows the scores and consequences for different events. If the hunter findsthe gold bar, s/he will receive 500 points and the game will end. If the hunterencounters the wumpus, s/he will lose 1000 points and the game will end. If the hunter8alls into a pit, s/he will lose 100 points but can still continue the game. The hunter willonly fall into a pit at the first time he encounters it. The hunter will get a 10-pointpenalty for uncovering every new location.TABLE 1:
Scores and consequences for different events
Event Score ConsequenceFind the gold bar +500 Map endsDiscover one new location -10 ContinueFall into a pit -100 Continue, no more points lost when revisitMeet wumups -1000 Map endsAn intelligent assistant helps the participant by recommending where to go. Theintelligent assistant is a knowledge-based agent and reasons using propositional logic(Russell & Norvig, 2010). Propositional logic is a mathematical model that reasonsabout the truth or falsehood of logical statements. By using logical inference, the agentwill give the values of four logical statements for a given location (e.g. location D P D, ; (2) there is no pit at this location,denoted as ¬ P D, ; (3) there is a wumpus at this location, denoted as W D, ; (4) there isno wumpus at this location, denoted as ¬ W D, . Based on the value of these 4 logicalstatements, we can categorize the location into one of the six different conditions shownin Figure 3: Y represents there is a pit/wumpus at this location (value of the first/thirdlogical statements is true); N represents there is no pit/wumpus at this location (valueof the second/fourth logical statement is true); NA represents the agent is not sureabout the existence of pit/wumpus at this location (values of all the four statements arefalse). The shaded squares in Figure 3 are the impossible cases because the pit andwumpus cannot co-exist in one location. For each case in Figure 3, the agent will assignprobabilities of encountering a wumpus, falling into a pit, finding a gold bar or nothinghappens as well as the corresponding expected scores if the hunter moves to thatlocation as shown in Table 2. The agent will randomly select one of the potential nextlocations with the highest expected score as the recommendation. 9D P(W, P, G, N) Expected Score1 (1 , , , − , , , − , , . , .
5) 2504 (0 , , , ) 133 .
335 ( , , , ) − .
676 ( , , , ) − option-centric rationale display proposed in this study. The option-centric rationale display details all the available next locations and the criteriafor choosing a particular location, and highlights the final recommendation using a redstar. The criteria for recommending a particular location depends on whether thehuman-autonomy team will find the gold bar, fall into a pit, encounter a wumpus, or 10 igure 4 . Testbed with option-centric rationale display.uncover a new location (without finding a gold bar, falling into a pit or encountering awumpus). The display also shows the possibility of each criterion and the correspondingexpected score. The display will group the next locations based on the criteria, i.e. iftwo locations have the same probabilities of each criterion, the display will list them inthe same row. The locations are sorted from the highest expected score to the lowest.The final recommendation is one of the locations with the highest expected score. Notethat the available next locations, the possibility of each criterion and the expectedscores are all computed by the intelligent assistant. The experiment used a within-subjects design. The independent variable in theexperiment was the presence/absence of the option-centric rationale display. The orderof the two conditions was counterbalanced to eliminate potential order effects. In eachcondition, participants played the game on 5 different maps. In the absence of the option-centric rationale display condition, the participant only saw a red star thatindicated the recommendation by the intelligent assistant. 11 .4 Measures
We measured three groups of dependent variables: subjective responses,behavioral responses and performance. After completing each map, participants wereasked to report their trust in the intelligent assistant and their self-confidence toaccomplish the task without the intelligent assistant using two 9-point Likert scales: (1)How much do you trust the intelligent assistant? (2) How confident are you incompleting tasks without the intelligent assistant? We calculated the recommendationacceptance as the rate that the participant followed the recommendations given by theintelligent assistant. Participants’ scores for each map were recorded as well.
In order to eliminate the inherent randomness of the task, we carefully selectedthe maps used in the experiment (Figure 5). First, we randomly generated 100 mapsand ran the game only with the intelligent assistant 20 times for each map (i.e, alwaysaccepted the recommendations from the intelligent assistant). We ranked the mapsbased on the standard deviation of the scores for each map from the lowest to thehighest. Second, we selected 10 maps which fulfilled three criteria: (1) Each map had alow standard deviation of the scores; (2) In each map, the gold bar was not just next tothe start location; (3) The locations of the gold bar in the 10 maps should be balancedacross the maps instead of concentrating in one part of the maps (e.g. upper rightcorner of the map). For each participant, the order of the 10 maps in the experimentare randomly determined. The second row in Table 3 shows the mean and standarderror of the intelligent assistant’s score of the 10 selected maps.We also developed 5 maps for the training session. Out of the 5 training maps,there are two maps with a pit next to the start location and three maps with lowstandard deviation of scores. The 5 training maps were presented according to thefollowing order: The first was similar to the maps participants experience in the realtest. Participants practiced on this map without the help of the intelligent assistant.The aim was to help participants get familiar with the game. From the second map 12nward, participants played the game with the help of the intelligent assistant. Thesecond and fourth practice maps were similar to the maps participants experienced inthe real test. The third and the fifth maps contained a pit next to the start location.The reason for selecting the two maps (i.e. the third and the fifth map) was to helpparticipants fully understand the stochastic nature of the game. For example, in thefifth training map (Figure 5), a breeze was detected by the sensor at the start locationand the two adjacent locations (i.e. B1 and A2) have the same probability of having apit. (a) Training map 1 (b) Training map 2 (c) Training map 3 (d) Training map 4 (e) Training map 5(f) Testing map 1 (g) Testing map 2 (h) Testing map 3 (i) Testing map 4 (j) Testing map 5(k) Testing map 6 (l) Testing map 7 (m) Testing map 8 (n) Testing map 9 (o) Testing map 10
Figure 5 . Selected maps for training and testing. First row: fixed order training map.Second and third row: testing map, order is randomly determined for each participant.13 .6 Procedure
All participants provided informed consent and filled in a demographics survey.After that, participants received a practice session. Participants played the game firstwithout the intelligent assistant, and practiced on another four maps with theintelligent assistant, and with or without the option-centric rationale display. In theexperiment, participants played the game with 5 maps in each condition. After eachmap, participants were asked to report their trust in the intelligent assistant and theirconfidence in accomplishing the game without the help of the intelligent assistant.Participants’ acceptance behaviors and task performance were recorded automaticallyby the testbed.
4. RESULTS
Data from 4 participants were discarded due to malfunction of the testbed. Datafrom 2 participants were discarded as their task performance were considered as outliersbased on the two-sided Dixon’s Q test (Dixon, 1953). All hypotheses were tested usingdata from the remaining 28 participants (Mean age = 21.25 years, SD = 1.72 years). Toanalyze the data, we performed structural equation modeling (SEM) using SPSSAMOS. SEM is a multivariate statistical method widely used in behavioral science(Bollen & Noble, 2011; Goldberger, 1972). It uses a path diagram to specify the model,which indicates the relationships between the variables. An arrow represents an effect ofa variable on another. In SEM, multiple regressions are simultaneously estimated toindicate the strength of each relationship (arrow). Measurement of fit in SEM providesinformation about how well the model fits the data. A good-fit model is required beforeinterpreting the causal paths of the structural model. option-centric rationale display, trust,self-confidence, recommendation acceptance and performance
To test hypotheses 1, 2 and 3, we constructed and tested a model as specified inFigure 6. Based on prior research (Hooper, Coughlan, & Mullen, 2008; Hu & Bentler,14999; Steiger, 2007), multiple model fit indices were used to check the model fit. Ourmodel demonstrated a good fit, with χ (6) = 3 . , p = .
71, RMSEA = 0 . .
00, SRMR = .
058 .The model is shown in Figure 6, with annotated standardized weights andsignificance (noted with asterisks). Standardized weights can be directly compared toassess the strength of each factor’s impact on the subsequent factor. In our study, thechains connecting display type, trust, recommendation acceptance, and taskperformance were significant: the option-centric rationale display increased participants’trust, which significantly affected their recommendation acceptance, and in turnimpacted the task performance. Yet, participants’ self-confidence did not affect theirrecommendation acceptance.
Figure 6 . The structural equation model indicates the relationship between the display,trust, self-confidence, acceptance and task performance.
To examine how trust changes overtime, we conducted a linear mixed effectsanalysis, with the presence/absence of the option-centric rationale display and trialnumber as fixed effects and intercepts for subjects as random effects. Results arereported as significant for α < . F (1 , . , p = .
018 (See Figure 7). Specifically, when participants were providedwith the option-centric rationale display, their trust in the autonomous agent increasedas they gained more experiences, F (1 , . , p < . F (1 , . , p = 0 . igure 7 . The change of trust over time.
5. DISCUSSION
In this study, we proposed an option-centric rationale display. The displayexplicitly conveys the intelligent assistant’s decision-making rationale by detailing allthe available next locations and the criteria for recommending a particular location.Our results indicate that with the option-centric rationale display, humanoperators trusted the intelligent assistant more. In addition, consistent with findingsfrom previous studies (Beller, Heesen, & Vollrath, 2013; Forster, Naujoks, Neukum, &Huestegge, 2017; Koo et al., 2014, 2016), we also found that trust significantly affectedpeople’s acceptance behaviors. As human operators’ trust in the autonomous agentincreased, they accepted more recommendations provided by the intelligent assistant.Meanwhile, human operators’ self-confidence did not influence their recommendationacceptance, which was in line with previous research (Lee & Moray, 1994; Moray et al.,2000). 16ABLE 3:
Mean and Standard Error (SE) values of the Intelligent Assistant’s Scoreand the Optimal Score for Each Test Map
Test Map ID 1 2 3 4 5 6 7 8 9 10Intelligent Assistant’s Score 450 ± ± . . ± . ± ± . ± . . ± . . ± . . ± . . ± . We found that higher recommendation acceptance led to higher task performance.We argued that this positive relationship hinged on the capability of the intelligentassistant, which was near-optimal in our study. Table 3 details the optimal score thatan omniscient agent could obtain and the score that the knowledge based intelligentassistant used in the present study could obtain. The optimal score was calculatedassuming that the intelligent assistant was omniscient (i.e., the map was known to theintelligent assistant). The intelligent assistant’s score was calculated by having theautonomous agent play the treasure hunter game by itself for 20 times. The intelligentassistant’s performance was close to the optimal score. The ratio between the intelligentassistant’s score and the optimal score was on average 91 . option-centric rationale display was presented. This implies that human operatorswould require less amount of interaction with the autonomous agent to calibrate theirtrust before trust reaching a steady state. This benefit is also attributable to theenhanced transparency provided by the option-centric rationale display. The list of allavailable options prevents severe drop of trust when optimal performance was notachieved simply because of the inherent randomness in the game.Although we only tested the option-centric rationale display on a simulated gamewith a small action space, the display can be applied to other decision-making agentswith a larger action space, for instance, an epsilon-greedy agent with finite (i.e.,countable) action space. The epsilon-greedy agent balances exploration and exploitation17y choosing the optimal action some times and the exploratory action other times. Theexploratory action is not the optimal action at a particular step. However, by furtherexploring the environment, the agent can obtain higher rewards in the subsequent stepsand higher accumulative rewards. The option-centric rationale display can list allpossible actions with the expected reward, and the number of times theoptimal/exploratory action has been taken to indicate the necessity of exploring theenvironment. For a large action space, the option-centric rationale display can present asubspace of the action space that contains the optimal and near optimal actions bylisting the actions with top expected scores. The other (far from optimal) actions canbe displayed if requested. Further research is needed to determine the size of subspaceto be displayed.
6. CONCLUSION
The advance in artificial intelligence and machine learning empowers a newgeneration of autonomous systems. However, human agents increasingly have difficultydeciphering autonomy-generated solutions and therefore perceive autonomy as amysterious black box. The lack of transparency contributes to the lack of trust inautonomy and sub-optimal team performance (Chen & Barnes, 2014; de Visser et al.,2018; Endsley, 2017; Lyons & Havig, 2014; Lyons et al., 2016; Yang et al., 2017). In thisstudy, we proposed an option-centric rationale display for enhancing autonomytransparency. The option-centric rationale display details all the potential actions andthe criteria for choosing a particular action, and highlights the final recommendation.The results indicate that the presence/absence of the display significantly affectedpeople’s trust in the autonomous agent and human operators’ trust increased fasterwhen the display was provided. The results should be reviewed in light of severallimitations. First, the intelligent assistant used in the present study was highly capable.However, in the real world, an intelligent assistant could be less capable in situations ofhigh uncertainty and ambiguity. Further research with less capable autonomous agentsis needed to validate the generalization of the display. Second, the action space in the18imulated game was limited. We discussed the application of the option-centricrationale display on domains with larger action space. Further research is needed toexamine the proposed solutions. 19eferencesBagheri, N., & Jamieson, G. A. (2004). The impact of context-related reliability onautomation failure detection and scanning behaviour. In (pp. 212–217). IEEE.Beller, J., Heesen, M., & Vollrath, M. (2013, March). Improving theDriver–Automation Interaction: An Approach Using Automation Uncertainty.
Human Factors , (6), 1130–1141.Bollen, K. A., & Noble, M. D. (2011, oct). Structural equation models and thequantification of behavior. Proceedings of the National Academy of Sciences of theUnited States of America , (SUPPL. 3), 15639–15646.Chen, J. Y. C., & Barnes, M. J. (2014, February). Human-Agent teaming formultirobot control: A review of human factors issues. IEEE Transactions onHuman-Machine Systems , (1), 13–29.Chen, J. Y. C., Procci, K., Boyce, M., Wright, J., Garcia, A., & Barnes, M. J. (2014). Situation awareness-based agent transparency (No. ARL-TR-6905) (Tech. Rep.).Aberdeen Proving Ground, MD.de Visser, E. J., Pak, R., & Shaw, T. H. (2018, April). From ‘automation’ to‘autonomy’: the importance of trust repair in human–machine interaction.
Ergonomics , (1), 1–19.de Vries, P., Midden, C., & Bouwhuis, D. (2003, jun). The effects of errors on systemtrust, self-confidence, and the allocation of control in route planning. International Journal of Human-Computer Studies , (6), 719–735.Dixon, W. (1953). Processing data for outliers. Biometrics , (1), 74–89.Du, N., Haspiel, J., Zhang, Q., Tilbury, D., Pradhan, A. K., Yang, X. J., & Robert,L. P. (2019). Look who’s talking now: Implications of AV’s explanations ondriver’s trust, AV preference, anxiety and mental workload. TransportationResearch Part C: Emerging Technologies , , 428–442.Du, N., Huang, K. Y., & Yang, X. J. (in press). Not All Information Is Equal: Effectsof Disclosing Different Types of Likelihood Information on Trust, Compliance and20eliance, and Task Performance in Human-Automation Teaming. Human Factors ,0018720819862916.Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003,June). The role of trust in automation reliance.
International Journal ofHuman-Computer Studies , (6), 697–718.Endsley, M. R. (2017). From Here to Autonomy: Lessons Learned FromHuman–Automation Research. Human Factors , (1), 5–27.Fletcher, K. I., Bartlett, M. L., Cockshell, S. J., & McCarley, J. S. (2017). Visualizingprobability of detection to aid sonar operator performance. In Proceedings of thehuman factors and ergonomics society annual meeting (Vol. 61, pp. 302–306).Forster, Y., Naujoks, F., Neukum, A., & Huestegge, L. (2017, December). Drivercompliance to take-over requests with different auditory outputs in conditionalautomation.
Accident Analysis and Prevention , , 18–28.Goldberger, A. S. (1972). Structural Equation Methods in the Social Sciences. Econometrica , (6), 979–1001.Hoff, K. A., & Bashir, M. (2015, April). Trust in Automation: Integrating EmpiricalEvidence on Factors That Influence Trust. Human Factors , (3), 407–434.Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling:Guidelines for determining model fit. Electronic Journal of Business ResearchMethods , , 53-60.Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structureanalysis: Conventional criteria versus new alternatives. Structural equationmodeling: a multidisciplinary journal , (1), 1–55.Kantowitz, B. H., Hanowski, R. J., & Kantowitz, S. C. (1997). Driver reliabilityrequirements for traffic advisory information. In Ergonomics and safety ofintelligent driver interfaces (p. 1-22). Lawrence Erlbaum Associates.Koo, J., Kwac, J., Ju, W., Steinert, M., Leifer, L., & Nass, C. (2014, April). Why didmy car just do that? Explaining semi-autonomous driving actions to improvedriver understanding, trust, and performance.
International Journal on nteractive Design and Manufacturing (IJIDeM) , (4), 269–275.Koo, J., Shin, D., Steinert, M., & Leifer, L. (2016). Understanding driver responses tovoice alerts of autonomous car operations. International Journal of VehicleDesign , (4), 377–17.Lee, J., & Moray, N. (1992). Tust contraol strategies and allocation of function inhuman-machine systems. Ergonomics , (10), 1243–1270.Lee, J., & Moray, N. (1994). Trust, self-confidence and operators’ adaptation toautomation. International Journal of Human-Computer Studies , (1), 153–184.Lee, J., & See, K. (2004). Trust in automation: designing for appropriate reliance. Human Factors , (1), 50–80.Lyons, J. B., & Havig, P. R. (2014). Transparency in a Human-Machine Context:Approaches for Fostering Shared Awareness/Intent. In R. Shumaker & S. Lackey(Eds.), Virtual, augmented and mixed reality. designing and developing virtual andaugmented environments (pp. 181–190). Springer International Publishing.Lyons, J. B., Ho, N. T., Koltai, K. S., Masequesmay, G., Skoog, M., Cacanindin, A., &Johnson, W. W. (2016). Trust-Based Analysis of an Air Force Collision AvoidanceSystem.
Ergonomics in Design , (1), 9–12. doi: 10.1177/1064804615611274MacLean, A., Young, R. M., Bellotti, V. M. E., & Moran, T. P. (1991, September).Questions, options, and criteria: Elements of design space analysis. Humman-Computer Interaction , (3), 201–250.Manzey, D., Reichenbach, J., & Onnasch, L. (2012, February). Human PerformanceConsequences of Automated Decision Aids: The Impact of Degree of Automationand System Experience. Journal of Cognitive Engineering and Decision Making , (1), 57–87.McBride, S. E., Rogers, W. A., & Fisk, A. D. (2011, nov). Understanding the effect ofworkload on automation use for younger and older adults. Human Factors , (6),672–686.McGuirl, J. M., & Sarter, N. B. (2006). Supporting trust calibration and the effectiveuse of decision aids by presenting dynamic system confidence information. Human actors , (4), 656–665.Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K.(2016). Intelligent agent transparency in human–agent teaming for multi-uxvmanagement. Human Factors , (3), 401-415.Miller, C. A. (2018). Displaced interactions in human-automation relationships:Transparency over time. In D. Harris (Ed.), Engineering Psychology and CognitiveErgonomics (p. 191-203). Cham: Springer International Publishing.Moray, N., Inagaki, T., & Itoh, M. (2000). Adaptive automation, trust, andself-confidence in fault management of time-critical tasks.
Journal of ExperimentalPsychology: Applied , (1), 44.Neyedli, H. F., Hollands, J. G., & Jamieson, G. A. (2011). Beyond identity:incorporating system reliability information into an automated combatidentification system. Human Factors , (4), 338–355.Oulasvirta, A., Feit, A., Lähteenlahti, P., & Karrenbauer, A. (2017). Computationalsupport for functionality selection in interaction design. ACM Transactions onComputer-Human Interaction , (5), 34:1–34:30.Parasuraman, R., & Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse,Abuse. Human Factors , (2), 230–253.Russell, S., & Norvig, P. (2010). Artificial Intelligence: A Modern Approach (3rd ed.).Pearson.Sheridan, T. B., & Verplank, W. L. (1978).
Human and computer control of underseateleoperators (Tech. Rep.). Cambridge, MA: Man Machine Systems Laboratory,MIT.Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structuralequation modeling.
Personality and Individual Differences , (5), 893–898.Sutcliffe, A. G., Poullis, C., Gregoriades, A., Katsouri, I., Tzanavari, A., & Herakleous,K. (2018, March). Reflecting on the Design Process for Virtual RealityApplications. International Journal of Human-Computer Interaction , (2),168–179. 23heodorou, A., Wortham, R. H., & Bryson, J. J. (2017). Designing and implementingtransparency for real time inspection of autonomous robots. Connection Science , (3), 230–241.Walliser, J. C., de Visser, E. J., & Shaw, T. H. (2016). Application of a system-widetrust strategy when supervising multiple autonomous agents. In Proceedings of theHuman Factors and Ergonomics Society Annual Meeting (Vol. 60, pp. 133–137).Wang, L., Jamieson, G. A., & Hollands, J. G. (2009). Trust and reliance on anautomated combat identification system.
Human Factors , (3), 281-291.Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017). Evaluating effects of userexperience and system transparency on trust in automation. In Proceedings of the2017 ACM/IEEE International Conference on Human-Robot Interaction (HRI’17) (pp. 408–416). New York, NY, USA: ACM.Yang, X. J., Wickens, C. D., & Hölttä-Otto, K. (2016). How users adjust trust inautomation: Contrast effect and hindsight bias. In
Proceedings of the HumanFactors and Ergonomics Society (Vol. 60, pp. 196–200). 24 iographiesRuikun Luo is a Ph.D. candidate at the Robotics Institute, University ofMichigan, Ann Arbor. Prior to joining the University of Michigan, he obtained a M.S.in Mechanical Engineering from Carnegie Mellon University in 2014 and a B.S. inMechanical Engineering and Automation from Tsinghua University, China in 2012.
Na Du is a Ph.D. pre-candidate in the Department of Industrial & OperationsEngineering at the University of Michigan. Her research interest includes trust inautomation and design of decision aids on trust calibration. Prior to joining theUniversity of Michigan, she completed a B.S. in Psychology in Zhejiang University,China in 2016.