Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery
EExplainable AI for Robot Failures: Generating Explanations thatImprove User Assistance in Fault Recovery
Devleena Das
Georgia Institute of TechnologyAtlanta, [email protected]
Siddhartha Banerjee
Georgia Institute of TechnologyAtlanta, [email protected]
Sonia Chernova
Georgia Institute of TechnologyAtlanta, [email protected]
ABSTRACT
With the growing capabilities of intelligent systems, the integrationof robots in our everyday life is increasing. However, when interact-ing in such complex human environments, the occasional failure ofrobotic systems is inevitable. The field of explainable AI has soughtto make complex-decision making systems more interpretable butmost existing techniques target domain experts. On the contrary,in many failure cases, robots will require recovery assistance from non-expert users. In this work, we introduce a new type of explana-tion, E πππ , that explains the cause of an unexpected failure duringan agentβs plan execution to non-experts . In order for E πππ to bemeaningful, we investigate what types of information within a setof hand-scripted explanations are most helpful to non-experts forfailure and solution identification. Additionally, we investigate howsuch explanations can be autonomously generated, extending an ex-isting encoder-decoder model, and generalized across environments.We investigate such questions in the context of a robot performinga pick-and-place manipulation task in the home environment. Ourresults show that explanations capturing the context of a failure and history of past actions, are the most effective for failure and solutionidentification among non-experts. Furthermore, through a seconduser evaluation, we verify that our model-generated explanationscan generalize to an unseen office environment, and are just aseffective as the hand-scripted explanations. CCS CONCEPTS β’ Human-centered computing β Interaction paradigms . KEYWORDS
Explainable AI, Fault Recovery
ACM Reference Format:
Devleena Das, Siddhartha Banerjee, and Sonia Chernova. 2021. ExplainableAI for Robot Failures: Generating Explanations that Improve User Assis-tance in Fault Recovery. In
Proceedings of the 2021 ACM/IEEE InternationalConference on Human-Robot Interaction (HRI β21), March 8β11, 2021, Boul-der, CO, USA.
ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3434073.3444657
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected].
HRI β21, March 8β11, 2021, Boulder, CO, USA Β© 2021 Association for Computing Machinery.ACM ISBN 978-1-4503-8289-2/21/03...$15.00https://doi.org/10.1145/3434073.3444657
In homes, hospitals, and manufacturing plants, robots are increas-ingly being tested for deployment alongside non-roboticists toperform complex tasks, such as folding laundry [49], deliveringlaboratory specimens [9, 25], and moving inventory goods [22, 33].When operating in such complex human environments, occasionalrobot failures are inevitable. When failures occur, human assistanceis often required to correct the problem [7], and co-located usersβ homeowners, medical lab technicians, and warehouse workers βwill be first on the scene. We classify such users as everyday users ,or non-experts , because of their lack of formal training in machinelearning, AI, or robotics.In order for everyday users to be able to assist in robot failurerecovery, they will need to understand why a failure has occurred.For example, a homeowner waiting for a robot to bring them coffeemay need to determine why the robot suddenly stopped in themiddle of the kitchen, or a production line worker may need todetermine why a packing robot suddenly stopped picking up items.The field of Explainable AI (XAI) has sought to address thechallenge of understanding "black-box" systems through the de-velopment of interpretable machine learning (ML) algorithms thatcan explain their decision making to users [2, 21]. Furthermore, asubfield of XAI, Explainable Planning (XAIP), has focused on gener-ating explanations specifically for sequential-decision making tasks,including explaining an agentβs chosen plan and explaining unsolv-able plans to end-users [12]. Such techniques hold great promise forthe development of more transparent robotic systems but they donot incorporate explanations for unexpected failures during a planexecution. Additionally, the majority of existing XAI techniquesare designed for technical domain experts who understand AI orML at its core [2, 40, 42, 48, 51]. While expert understanding iscrucial, such XAI methods are not suitable for the vast majority ofend-users, who are non-experts [18, 20, 26].In this work, we seek to make robotic systems more transparentto their users by leveraging techniques from explainable AI, whilealso extending the capabilities of XAI systems toward greater trans-parency for non-expert users. Specifically, our work addresses faultrecovery cases in which the robotβs task execution is halted due toan error. We investigate whether providing explanations can notonly help non-expert users understand the systemβs point of failure,but also help them determine an appropriate solution required toresume normal operation of the task.In some cases (e.g., complexhardware failures), the user might not have the knowledge to fix thepoint of failure regardless of the provided error explanation. In thiswork, we focus on failures that we expect to be within the userβsunderstanding (e.g., object is too far away), and we address this a r X i v : . [ c s . A I] J a n uestion in the context of pick-and-place manipulation tasks in thehome environment. Our work makes the following contributions: β’ Formalization of error explanations:
Providing justifica-tions for points of failures that occur unexpectedly amidst anagentβs plan execution has not previously been studied withinthe XAIP community. We expand upon the existing set of expla-nations available in the XAI and XAIP community, introducing error explanations designed to explain failures that occur duringthe execution of a task. β’ Explanation content:
We empirically evaluate what informa-tion an error explanation should contain to aid non-experts inunderstanding the cause of failure and to select a recovery strat-egy. We show that explanations that include both (i) historyof recently accomplished actions, and (ii) contextual reasoningabout the environment, are the most effective in enabling usersto identify the cause of and solution to a failure. β’ Explanation generation:
We present an automated techniquefor generating natural language error explanations that ratio-nalize encountered failures in a manner that is understandableby non-experts. Specifically, we extend an encoder-decodermodel for autonomously generating natural language expla-nations introduced by [20] to generate context-based historyexplanations within a continuous state-space. β’ Validation with non-expert users:
We demonstrate that ex-planations generated by the encoder-decoder model can gen-eralize to an unseen environment and are as effective as hand-scripted, context-based history explanations.We validate our approach through two user studies and com-putational model analysis. In the first study, we examine whatinformation an error explanation should contain by evaluating howthe content of an explanation affects user performance in identify-ing and assisting with a robot error (Sec. 4). From these results, weidentify an explanation type that leads to the highest performance,and then contribute a computational model to automatically gener-ate such explanations from robot states (Sec. 5). In our second study,we demonstrate that our automated explanations are as effectiveas hand-scripted explanations in guiding non-experts to identifythe cause of a failure and its potential solution.
The XAI community has primarily focused on developing inter-pretability methodologies for understanding the inner workings ofblack-box models [2, 40]. Many of these approaches have focusedon model-agnostic implementations, designed to increase expert understanding of deep learning outputs [2, 38, 39]. Additionally,most XAI approaches [40, 42, 48, 51] have primarily focused onunderstanding classification-based tasks. However, classificationtasks do not capture the complexity of sequential decision-makingan agent, such as a robot, may perform while having long-terminteractions with users [12].To address the need for interpretable explanations in sequentialdecision making tasks, the XAIP community has focused on explain-ing an agentβs plans to end-users. A recent survey paper highlightssome of the key components of plan explanations studied by thecommunity [12]: (1) contrastive question-answering, (2) explain-ing unsolvable plans, and (3) providing explicable justifications for a chosen plan. In the realm of contrastive question-answering,Krarup et al. provide a framework to transfer domain-independentuser questions into constraints that can be added to a planningmodel [32], while Hoffmann et al. utilize common properties withina set of correct plans as an explanation for unmet properties in in-correct plans [24]. In order to explain unsolvable plans, Sreedharanet al. abstract the unsolvable plan into a simpler example throughwhich explanations can be formulated [43]. Finally, in order to pro-vide explicable justifications for a plan, Zhang et al. use conditionalrandom fields (CRFs) to model human explanations of existing agentplans, and use such a human βmental modelβ as a constraint for gen-erating explicable plans [52]. The work is extended by Chakrabortiet al., who eschew constraining an agentβs plan and instead achieveexplicability through model reconciliation, whereby the agent pro-vides explanations that reconcile its model to the human βmentalmodelβ [11, 13]. However, in these works, an explanation justifiesa chosen plan, or the lack of one. In our work, we aim to explainthe possible failures that can arise during a plan.Techniques for plan repair enable a task plan to be adaptedto overcome an error [14, 23]. Methods in this domain reuse thefailing plan and search the plan space to find local deviations thatallow continued execution [15], or transform the plan to adapt itto the situation [27, 34]. Such repairs are often found and executedautonomously, with no human intervention.Recently, works have considered interactive plan repair witha human-in-the-loop. Boteanu et al. showed a proof-of-conceptmodel in which a human approved repair action is proposed bya common-sense reasoning framework [10]. However, this workwas limited to errors involving missing items, and did not focus onexplaining errors to users. Meanwhile, Knepper et al. investigatedthe grounding of natural language requests to best garner help fromnon-expert humans [31]. They found that the requests were suc-cessful when they were targeted, e.g. helped listeners disambiguatebetween multiple objects, and told them what to do. The authorsdeveloped a system to generate such requests. We build upon thesefindings to investigate the characteristics of natural language errorexplanations that allow non-experts to help a robot; unlike [31], wedo not assume the robot is aware of a correct recovery and able todirect the user on the recovery process.Plan repair for failures that occur during execution require afault diagnosis [23] and there is a large body of ongoing work inrobotics focused on fault diagnosis techniques [29]. These worksuse a range of methods, including unsatisfied preconditions [10,15, 31], first-order logic inference [50], case-based reasoning [23,36], sensor signal processing [1, 17, 28], Bayes nets and DynamicBayesian Networks [8, 30], Hidden Markov Models (HMMs) [47],particle filters [44, 53], and neural networks [35, 37] to diagnosefailures. Depending on the context of the work, the diagnosis eitheridentifies what is wrong with the robotβe.g., object not visible [31]or sonar is blind [17]βor why it is wrongβe.g., there was a collisionwith the environment [35]. However, the prior work aims to usethe diagnoses for autonomous robot recovery from failure or tofacilitate debugging by experts. The problem of generating naturallanguage explanations from a fault diagnosis to allow non-expertsto help a robot recover remains largely unexplored.In efforts to provide explanations to non-experts on infeasibleagent behaviors, prior work has presented a linear temporal logic igure 1: The pipeline used to generate E πππ explanations for a non-expert user. (a) Data collection in failure simulations andthe extraction of the agentβs state space. (b) Study of hand-scripted E πππ explanations with varying information types (Sec. 4).(c) Autonomously generated E πππ explanations using an encoder-decoder model (Sec. 5). (LTL) framework to explain actions unsatisfiable by a robot [39].The explanations focus on what actions are unattainable by therobot, but do not include the underlying reasons for why they maybe unattainable. Similarly, an algorithm called HIGHLIGHTS usesvisual animations to summarize agent capabilitiesβ what an agentcan achieveβto non-expert users, based on the features dictating anagentβs reward function [3]. In our work, we show that non-expertsneed to be told why an action is unattainable in order to help anagent recover; explaining what is unattainable is insufficient.Finally, prior work in XAI has found that natural language expla-nations can provide βjustificationβ and are βunderstandableβ by non-experts [20]. The study, conducted in the discrete domain of Frogger,used sequence-to-sequence learning to treat explanation generationas a neural translation problem, where an agentβs internal statesare translated into natural language, with impressive results onnon-expertsβ abilities to comprehend the agentβs decisions [19, 20].We build on these findings and adapt the sequence-to-sequencelearning approach to a continuous robotics domain. We define the problem of providing explanations for task failures byextending the framework introduced by Chakraborti et al. [12] forproducing explanations to goal-directed plans. In the framework, aplanning problem Ξ is defined by a transition function πΏ Ξ : π΄ Γ π β π Γ R , where π΄ is the set of actions available to the agent, π is theset of states it can be in, and R is a cost of making the transition. Aplanning algorithm A solves Ξ subject to a desired property π toproduce a plan or policy π , i.e. A : Ξ Γ π β¦β π . Here, π may representdifferent properties such as soundness, optimality, etc. The solutionto this problem, i.e. the plan , π = β¨ π , π , ..., π π β© , π π β π΄ , whichtransforms the current state πΌ β π of the agent to its goal πΊ β π , i.e. πΏ Ξ ( π, πΌ ) = β¨ πΊ, Ξ£ π π β π π π β© . The second term in the output denotes aplan cost π ( π ) .Given the above framework, we define two explanation types.The first is from [12], and the second is contributed by our work: E π : This explanation justifies to a human user that solution π satis-fies property π for a given planning problem Ξ . For example,the user may ask βWhy π and not π β² ?β. In response to thisquestion, E π must enable the user to compute A : Ξ Γ π β¦β π and verify that either A : Ξ Γ π ΜΈβ¦β π β² , or that A : Ξ Γ π β¦β π β² but π β‘ π β² or π is greater than π β² with respect to some criteria. E π applies to the plan solution as a whole and can be elicitedat any time. Approaches addressing E π are discussed in Sec. 2. E πππ : This explanation applies when an unexpected failure state, π βF , is triggered by a failed action in β¨ π , π , ..., π π β© , and halts theexecution of π . For example, the user may ask βThe robot is atthe table, but why did it not pick up my beverage?β In responseto this question, E πππ must allow the user to understand thecause of error in order to help the system recover.In this work, we develop the second variant of explanations, E πππ .We assume that both the algorithm A and the plan π are sound,and that the cause of error is triggered by a failure state π β F from which an agent cannot recover without user assistance. Forexample, a situation in which a robot requires human help to discernan occluded object or pickup a tool out of reach. Our objective isto generate E πππ such that the user (1) correctly understands thecause of failure, and (2) helps the agent recover from the error byproviding a solution.In the following sections, we present our methods to achieve theabove objective. In Sec. 4, we introduce a set of information types , Ξ ,that are characteristics of E πππ . We then develop scripted explana-tions satisfying different π β Ξ , and evaluate them to find a mean-ingful π that satisfy our objective for non-expert users (Fig. 1(b)).The results from Sec. 4 inform our efforts in Sec. 5 to automaticallygenerate E πππ without using pre-defined scripts (Fig. 1(c)). E πππ In order to generate E πππ , the first question we have to answer is: given an error while executing a plan π for a particular task, whattypes of information should explanation E πππ contain ? tudy Condition π π‘ π π‘ β π π‘ Example Explanation for βobject is occludedβ failure
None N/AAction Based (AB) β Robot could not find the object.
Context Based (CB) β β
Robot could not find the object because the object is hidden from view.
Action Based History(AB-H) β β
The robot finished scanning objects at its current location but could not find the desired object.
Context Based History(CB-H) β β β
The robot finished scanning objects at its current location, but could not find the desired objectbecause the desired object is hidden from view.
Table 1: The features that can encompass an explanation based on the study conditions. π π‘ represents current action, π π‘ β represents last successful action, and π π‘ represents captured environmental context. For our application, we desire that E πππ is (1) accessible to non-experts, and (2) representative of the fault cause. Unfortunately, itis not clear from prior literature what information from an agentβsplan π , or its failure, satisfies these requirements. Ehsan et al. [20]propose that explanations for everyday users should take the formof rationales , which justify the agentβs decision in laypersonβs terms.However, their rationales are trained from non-expert labels, donot reveal the true decision making process of an agent, and thuswould not be able to disambiguate among visually similar roboterrors (e.g., failure to grasp object due to kinematic constraints vs.object occlusion vs. a segmentation error). By contrast, prior workon fault diagnosis [29] has extensively studied how to describeerror states, but such work exclusively targets expert users, withresulting explanations referencing specific system components oragent internals (e.g., βlocalization mismatch with odometryβ [17]).Thus, our first step is to determine what information E πππ shouldcontain to be both accurate and interpretable by non-experts.In this section, we define a set of information types, π β Ξ , thatwe use to generate scripted explanations during a failure. In a userstudy with non-experts, we determine which π best help usersidentify the cause of a failure and suggest solutions to the failure.Specifically, we conducted a between-subjects user study in which Ξ consists of four values that are a cross-product of two factors:2 (history, no history) x 2 (context-based, action-based). In the userstudy, the four explanation conditions were contrasted against abaseline condition. The five conditions are enumerated below: β’ Baseline (None) : Participants receive no explanation on thecause of error. This is the current standard in deployed roboticsystems, e.g., [41]. β’ Action-Based (AB) : Participants receive E πππ containing onlythe currently failed action π π‘ as the cause of error. β’ Context-Based (CB) : Participants receive E πππ containingboth π π‘ and context, π π‘ , retrieved from the environment asthe cause of error. β’ Action-Based-History (AB-H) : Participants receive E πππ con-taining the previous action π π‘ β and π π‘ as the cause of error. β’ Context-Based-History (CB-H) : Participants receive E πππ containing π π‘ β , π π‘ , and π π‘ as the cause of error.Table 1 summarizes the study conditions and provides example ex-planations for each condition. In the following sections, we discussour experimental setup, the study procedure, and our results. We conduct our experiment in Gazebo simulations of a Fetch ro-bot [46]. The Fetch robot is a mobile manipulator with a differential drive base, a 7DoF arm, a parallel-jaw gripper, a pan-tilt head, andan adjustable torso. For sensing, the base includes a laser scannerand the head contains an RGB-D camera. The robot is simulatedin a kitchen setting performing a pick-and-place task (as seen inFig. 1a). The robotβs task is to move a task-specified object (e.g.,milk carton) from the dining table to the kitchen counter.Similar to prior work in robotics [5], we define the robotβs actionspace as the set π΄ = { πππ£π, π ππππππ‘, πππ‘πππ‘, π πππππππ π, ππππ π, ππ π π‘,πππππ } , where πππ£π navigates the robot to a specified location, π ππππππ‘ is used to identify which pixels in its sensory space corre-spond to objects, πππ‘πππ‘ performs object detection to obtain a labelfor a given object, π πππππππ π executes grasp sampling to identifypossible grasp poses for the gripper, ππππ π moves the robot arminto a grasp pose and closes the gripper, ππ π π‘ raises the arm, and πππππ places a held object at a specified location.The robotβs state at each time step π‘ is defined as π π‘ β π , where π = π π βͺ π π βͺ π π βͺ π π . Here, π π = π π βͺ π π denotes the set of namesfor all entities in the environment, where π π consists of {milk, cokecan, ice cream, bottle, cup} , and π π consists of: {dining table, kitchencounter} . π π ( π‘ ) β π π is a vector of β¨ π₯, π¦, π§ β© locations for each en-tity π π β π π at a given time step π‘ . π π ( π‘ ) β π π is defined by threetuples β¨ π₯ ππ£ππ , π¦ ππ£ππ , π§ ππ£ππ β© , β¨ π₯ ππ£ππ , π¦ ππ£ππ , π§ ππ£ππ β© , β¨ π₯ πππ , π¦ πππ , π§ πππ β© thatdescribe the angular velocity, linear velocity and position of the ro-bot at π‘ . Finally, π π = { π ππππ π , π π πππππππ π , π πππ£π , π ππππ , π πππ‘πππ‘ , π π ππ } where π π ( π‘ ) β π π describes the status of each π β π΄ at π‘ , and whethereach action is: active ( ), completed ( ) or errored (- ). Therefore,at all time steps, the number of elements in π π ( π‘ ) is equal to thenumber of actions in π΄ . The agentβs initial state is defined as π = {β¨ , , β© , β¨ , , β© , β¨ , , β© , { ππ’ππ }} , where the position tuple and the velocity tuples are set tozero, and the action states π π ( ) are not defined. If there are no er-rors, the agentβs final state is defined as π π = {β¨ π₯ π , π¦ π , π§ π β© , β¨ , , β© , β¨ , , β© , { , , ..., }} , where the position tuple is set to the goal lo-cation, the velocity tuples are zero, and each action state in π π ( π ) is 1. In this context, plan π is the set of actions β¨ π , π , ..., π π β© β π΄ that transform the agentβs initial state π to its final state π π . Wethen define a failure π in plan π as the event when any action statein π π has a value -1.Following the example of prior work [35], we study our work inthe context of a representative sample of failures in robot behavior.We classify these failures using fault-tree analysis (Fig. 2) . Failed Our fault-tree analysis identifies errors and solutions relevant to our domain, and weleave generating explanations for unknown errors for future work. Visualization ofeach failure is available at: https://youtu.be/jYn3FaqG65E. igure 2: Fault tree analysis of failures in this work. We alsoshow the failed action used to detect the failure, and a short-hand label of the solution to fix the failure. robot behaviours characterize coarse failure types ( πΉ π‘ ), e.g., a failurein βobject detectionβ. Each failure type can have multiple failurecauses ( πΉ π ), e.g., βobject not presentβ or βobject occludedβ are possi-ble causes for an βobject detectionβ failure. In our system, failuresare detected by an errored action. For example, π π πππ‘πππ‘ = β canindicate that either the βobject is occludedβ or the βobject is notpresentβ. Crucially, however, each failure cause has an associatedresolution action, not in the robotβs action space, but which can beselected by humans to rectify the cause of failure.The fault-tree analysis also groups failure causes πΉ π into causalgroups, which we define as Internal and
External . These categoriesroughly correspond to system and environment failures, respec-tively, in the prior work [35]. Internal failures are not apparentthrough visual cues in the environment and are often the resultof failures of hardware or software modules. By contrast, externalfailures are often caused by unexpected conditions in the environ-ment and are therefore visually apparent in the environment. InSection 4.5, we investigate the effect of different information typeson users when the error stems from the different causal groups.
Our objective is to evaluate the different information types of errorexplanations across a variety of failures. We simulated |F | = | π π | Γ| πΉ π | = failures to capture all possible object Γ failure causecombinations. In our domain, each failure π β F has a singlecause in πΉ π and therefore a single resolution method πΉ π . The studyconsisted of the following three stages. Familiarization:
Participants in all conditions were first shownthree videos of the Fetch robot successfully executing the task withrandomly selected objects from π π using a plan π . This served toaccustom participants to the robot, its abilities, and its actions. Baseline:
All participants were then shown six randomly sam-pled failure simulations from F , one for every failure cause πΉ π .To visualize the failure, participants were shown animated snap-shots (GIFs) of actions leading up to a failure, and three perspectiveshots of the robot in the final environment state . Participants wereprovided no explanations and asked to identify the cause of thefailure and suggest a solution. Participant responses established the Humans subject study is available here: https://robotasks00.web.app/. participantsβ baseline understanding of the robot and the domain,allowing us to measure improvement in understanding.
Explanation:
Finally, participants were exposed to twelve addi-tional randomly sampled failures from F (different from Baseline ),two for every failure cause πΉ π . Depending on the assigned studycondition, a participant was either provided a hand-scripted ex-planation matching the information type of the assigned studycondition, or the participant was provided no explanation if in the None condition. As before, the participant was required to identifythe cause of failure and suggest a solution. For each simulation,after identifying a failure and solution, participants received theiraccuracy score. This was the only feedback given to all participants.
We evaluate participant performance using F1 score. In particular,we evaluate the difference between participant
Baseline
F1 score andtheir
Explanation
F1 score. The difference in F1 score is evaluatedfor the following measures: β’ Failure Identification (
FId ) : measures a participantsβ abilityto correctly identify the cause of each failure. β’ Solution Identification (
SId ) : measures a participantsβ abilityto correctly identify the solution to each failure.Our data analysis then aims to answer the following questionswith respect to the measures: β’ Q1 : Do action-based (AB) or context-based (CB) explanationslead to the greatest improvement in user failure identification( FId ) and solution identification (
SId )? β’ Q2 : Does the inclusion of history within an explanation im-prove usersβ failure identification ( FId ) and solution identifica-tion (
SId )? β’ Q3 : How do usersβ failure identification ( FId ) and solution iden-tification (
SId ) compare for Internal vs External robot errors?
Participants . We recruited 80 individuals from Amazonβs Mechan-ical Turk. Since our target audience is non-experts, we filtered out10 participants for achieving 100% accuracy in the
Baseline stage,under the assumption that they were not novices. The remaining70 participants included 51 males and 19 females, all whom were18 years or older (M = 35.2 , SD = 9.4). Due to the exclusion crite-ria, each study condition had 13-15 participants. The task took onaverage 20 - 40 minutes and participants were compensated $3.50.
Data Analysis . The data on the
FId and
SId metrics are analyzedwith a two-way ANOVA for Q1 and Q2 and a one-way ANOVAfor Q3 , followed by a Tukey HSD post-hoc test for each. Fig. 3a and Fig. 3b answer Q1 by showing the benefit of includingenvironmental context (CB, CB-H conditions) in failure identifi-cation (
FId ) and solution identification (
SId ). In both figures, wesee that explanations with context have the highest improvementin
FId and
SId scores. Specifically, the presence of context had asignificant effect on
FId (F(2,67)= 6.95, p=0.0018), with a significant
FId improvement for Context-based explanations over both None(t(67)=3.729, p=0.0012) and Action-based (t(67)=2.923,p=0.014) ex-planations. Similarly, the presence of context had a trending effecton
SId (F(2,67)=2.92, p=0.06), with a significant improvement in a) (b) (c) (d)
Figure 3: Average F1 score across explanation conditions grouped by Context Based vs. Action Based (a-b) and History vs. NoHistory (c-d). In Fig. 3, 4, and 6, statistical significance is reported as: *p < 0.05, **p < 0.01, ***p < 0.001 (a) (b) (c) (d)
Figure 4: Average F1 score across all conditions grouped by Internal versus External errors.
SId for Context-based explanations vs. None (t(67)=3.12, p=0.007).This indicates that the inclusion of environmental context in theCB explanation conditions (CB, CB-H) helped participants betterunderstand the underlying causes of the failures thereby allowingthem to better assist the robot.
Fig. 3c and Fig. 3d answer Q2 by showing the benefit of includinghistory (AB-H, CB-H conditions), on
FId and
SId . In both figures,history-based explanations have the highest improvement in
FId and
SId scores. Similar to the effects of including context, includ-ing history had a significant improvement on
FId (F(2,67)= 3.36,p=0.04), with History vs. None as significant (t(67)=3.447, p=0.003).Although including history did not have a significant effect on
SId overall (F(2,67)= 1.38, p=0.25), we observe a significant differ-ence in improvement between History-based explanations vs. None(t(67)=3.1857, p=0.006). This supports the idea that knowledge ofthe most recently completed action (AB-H, CB-H conditions) canhelp users gauge what a robot was able to successfully accomplish,thereby helping users better pinpoint the exact cause of failure andprovide correct suggestions for recovery.Our analysis so far investigates the independent effects of includ-ing context and history on explanation utility. The results suggestthat context-based explanations incorporating history, i.e. CB-H ex-planations, are the best suited to non-experts. We next consider eachexplanation type individually and their efficacy for non-expertsbased on the causal group of the originating fault.
Fig. 4 answers Q3 by showing the different effects of the explana-tion types for failures stemming from the different causal groupsβ
Internal and
External failures. Explanations have a significant effect on the improvement in
FId for
External errors (F(4,62)= 3.53, p=0.01),with CB-H showing the most pronounced improvement, specificallyvs. AB (t(62)=-3.216, p=0.017) and vs. None(t(62)=-3.046, p=0.027).Additionally, we see a significant effect of explanations in improv-ing
FId for
Internal errors ((F(4,62)= 4.39, p=0.003), with a significantdifference in CB-H vs. None (t(62)=-3.955, p=0.0018). With respectto improvement in
SId for
External errors, we see a trending effectof explanations (F(4,62)=2.16, p=0.083) with a trending difference be-tween AB-H vs. None (t(62)=2.648, p=0.073). For
Internal errors, wenotice a trending effect of explanations (F(4,62)=2.37, p=0.061), butwith a significant difference between CB-H and None (t(62)=-2.86,p=0.044). Overall, we find that CB-H explanations are valuable toparticipants for both error types, but especially for the Internal casewhen failure causes are not discernible through the environment. E πππ In Sec. 4 we discovered CB-H explanations to be most effective. Inthis section, we introduce an automated explanation generationsystem that can generate the CB-H explanations word by word,without a template. We adapt a popular encoder-decoder network [4, 6] utilized by [20]to train a model to generate CB-H explanations from an agentβs state.The modelβs features, π , are derived from the state space, π (seeSec. 5.2), and are comprised of environment features π , continuous The system is also able to generate AB, AB-H, and CB explanations; but we focus onCB-H due to its highest
FId and
SId scores in Sec. 4.5. igure 5: Confusion matrix analysis of our modelβs perfor-mance where the first six columns represent E πππ explana-tions and the last column represents E ππππ rationalizations.The x-axis represents the true labels, and the y-axis repre-sents the predicted labels. features, π , and a desired object of interest, π . The encoder receivesthe environment features as input and produces an embedding ofthe environment context in its hidden state, β π . The embedding isthen appended to the continuous features and the object of interest,and the concatenated features are given to the decoder as input. Thedecoder generates a sequence of target words, π = { π¦ , π¦ ...π¦ π } ,where π¦ π is a single word, and π is the CB-H explanation. The modelarchitecture is shown in Fig. 1(c).The encoder and decoder are comprised of Gated RecurrentUnits ( πΊπ π ) [16]. Given a sequence of environment features, π = { π₯ , π₯ ...π₯ π } , the encoder generates the context embedding at se-quence step π by, β π = πΊπ π ( π₯ π , β π β ) , where β π β is the previousstepβs context embedding. The decoder uses the final context em-bedding, β π , concatenated to the continuous features, π , and theobject of interest, π , as its initial input, π . The decoder also gener-ates and uses a weighted attention vector, π π , for step i (initializedwith π = ). At each step, π π attends over the features in π and π π β ,the decoderβs input at the previous step. The decoder then updatesits state according to the function π π = πΊπ π ( π π , π¦ π β , π π ) , where π¦ π β is the previous predicted word, and a word, π¦ π , is predicted fromthe maximum softmax probability over π π . A complete explanationis generated when the decoder predicts the βENDβ token. Recall from Sec. 4.1 that the agentβs state space is defined as π = π π βͺ π π βͺ π π βͺ π π . We derive the features, π = π βͺ π βͺ π for the encoder-decoder model from π . The object of interest, π β π π , is specifiedas part of the task and represented by its word embedding in π .The environment, π , is comprised of the word embeddings of thenames of the objects, ππ π πΊ , located in the robotβs area of interest,such that β π β² β ππ π πΊ , π β² β π π . The remaining continuous features, π = { π ππ π β πΊπππ , π ππ π β π , π£ πππ , π£ πππ , π π , π ππ π β ππ π πΊ , π π } , characterize the robot and the target object in the environment. π ππ π β πΊπππ is thedistance of the robot from its goal location,
π ππ π β π is the distance ofthe robot from the target object, π£ πππ and π£ πππ are the angular andlinear velocities of the robot base, and π π are the action statuses asdefined in Sec. 4.1. Additionally, π ππ π β ππ π πΊ is the distance betweenthe desired object π and the objects in ππ π πΊ , and π π is a booleanthat evaluates to true if π β ππ π πΊ . Note that not all the features in π contain valid values at all times. If a feature value is invalid, thefeature is masked before it is concatenated into π . To further evaluate the generalizability of our method across envi-ronments, we expand our data to include simulations from an officeenvironment (Fig. 1(a)), in addition to the kitchen environment in-troduced in Sec. 4. The office environment contains different objectsand locations, i.e. entities π π , but the robotβs pick-and-place taskremains the same. Entities in the office environment have a one-to-one correspondence to the entities in the kitchen environment.Our dataset π· consists of 72 simulations (60 and 12 from thekitchen and office environments, respectively). Each timestep in π· is defined by π’ π‘ , where π’ π‘ β π represents the input features to ourencoder-decoder model at timestep π‘ . Each simulation begins with π active or successful action timesteps, denoted by π π ( π‘ ) = or π π ( π‘ ) = , and ends with π error timesteps, denoted by π π ( π‘ ) = β .In our work, π ranges from 15 to 20, and π = . The π errortimesteps simulate a robot repeatedly attempting to autonomouslyremedy a failure upon encountering it, reflecting a real world solu-tion to errors where robots try to repeat actions that fail [5].Given our dataset, we annotate error timesteps with a CB-Hexplanation, E πππ , and annotate successful or active timesteps witha natural language rationalization of the state, E ππππ , as in [19]. Inour work, examples of such rationales include, βrobot moving todining tableβ and βrobot segmented objects in the sceneβ. Addition-ally, E ππππ explanations were only used for model training, andwere not a focus of the human subjects study in Sec. 5.5. The totalsize of π· is 2100 timesteps where there are 1380 successful or activetimesteps, and 720 error timesteps. Our encoder-decoder model is trained with the 60 kitchen sim-ulations using a two-step grouped leave one out cross validation(LOOCV) with 10 folds, where the grouped LOOCV leaves out an en-tire simulation for each failure cause, πΉ π . The first grouped LOOCVcreates a split between the training set, π π‘π , and test set, π π‘π , whilethe second LOOCV creates the validation set, π π£ . As a result, in eachfold, π π‘π includes 48 simulations with 480 error explanations, while π π£ and π π‘π include 6 simulations, each with 60 error explanations.To evaluate each fold, we utilize an evaluation set, π ππ£ππ , whichincludes the 12 office simulations with 120 error explanations. Training.
Our models trained for an average of 180 epochs, de-pending on the validation loss. We train with a batch size of 20.Our GRU cells in the encoder have a hidden state size of 20 and theGRU cells in the decoder have a hidden state size of 49. We trainour model using a Cross Entropy loss optimized via Adam with alearning rate of 0.0001.
Evaluation . Fig. 5 shows the average performance of the model on π ππ£ππ across the 10 folds of cross-validation. The confusion matrix a) (b) Figure 6: Average F1 scores between participants who re-ceived model generated explanations (CB-H-M), scripted ex-planations (CB-H), and no explanations (None). includes accuracy on explanations, E πππ , for the six failure causesas well as accuracy on the non-error rationalizations, E ππππ . Anexplanation or rationalization is marked correct only if it identicallymatches its target phrase.On average, our model can generalize explanations across thesix failure causes with 81.81% accuracy. For each failure scenario,the model has a larger true positive rate than false positive rateor false negative rate. We observe that the model is accurate indetermining βarm motion planningβ failures but struggles to differ-entiate between its causes: βobject too far awayβ and βobject tooclose to othersβ (Fig. 2). We also notice that explanations of theβnavigationβ and βobject detectionβ failure types sometimes indi-cate causes they are not associated: e.g., βcontroller errorβ wronglypredicted as βobject too far awayβ or βobject too close to othersβ,or βobject occludedβ wrongly predicted as βobject too far awayβ orβobject too close togetherβ. We suspect that the challenges stemfrom our modelβs continuous feature space, making certain featuresharder to distinguish and that with additional training data, thegeneralizability of our model can be improved. Model Selection.
Of the 10 models trained with LOOCV, we se-lected the best model based on its performance on π ππ£ππ . The bestmodel was deployed in a user evaluation described below. We conducted a user evaluation similar to the one described inSec. 4. The study was a three condition between subjects study,where participants were either provided with no explanations oferrors (
None ), context-based-with-history scripted explanations(
CB-H ), or context-based-with-history model-generated explana-tions (
CB-H-M ). During the study, participants were shown kitchensimulations in the
Baseline portion of the study, and evaluated onthe 12 office simulations in the
Explanation portion of the study.
Hypotheses.
We wished to evaluate whether (1) the model gen-erated explanations improved participantsβ failure and solutionidentification compared to the None condition, and (2) the modelgenerated explanations performed on par with the hand-scriptedexplanations in improving participantsβ performance.
Participants.
We recruited 45 individuals from Amazonβs Mechan-ical Turk. After applying the exclusion criteria as before, the re-maining 41 participants included 25 males and 16 females, all whomwere 18 years or older (M = 39 , SD = 11.3). Due to the exclusion criteria, each study condition had 12-15 participants. The task tookroughly 20 - 40 minutes and participants were compensated $3.50.
Data Analysis.
The data on
FId and
SId metrics are analyzed witha one-way ANOVA followed by a Tukey HSD post-hoc test.
Fig. 6 answers our hypotheses by showing that CB-H-M explana-tions are just as effective as CB-H scripted explanations. We observea significant effect of explanations on
FId (F(2,38)=10.52, p=0.0002)and
SId ((F(2,37)=3.94, p=0.027). With respect to
FId we observe asignificant improvement in participant accuracy between CB-H-Mand None (t(38)=-4.158,p=0.00049), and no significant differencebetween CB-H and CB-H-M (t(38)=-0.208, p=0.97). With respectto
SId we observe a trending difference in improvement betweenCB-H-M vs. None (t(37)=2.354, p=0.060), a significant differencebetween CB-H vs, None (t(37)=2.561, p=0.038) and no significantdifference between CB-H and CB-H-M (t(37)=0.215, p=0.974). Thus,we conclude that given CB-H-M explanations, participants performjust as well in helping the robot as when given CB-H explanations.
In this work, we investigate what types of information within an ex-planation help non-experts identify robot failures and help assist inrecovery. We introduce a new type of explanation, E πππ , which hasnot been previously addressed in the XAIP community, and whichdescribes the cause of unexpected failures amidst plan execution.Our results indicate that for explanations to improve failure andsolution identification, they should encompass both environmen-tal context and history of past successful actions. Furthermore, inour first user evaluation we showcase the importance that context-based-history explanations serve in the cases of Internal errors,which are not visually observable through environmental changes.Additionally, we investigate a method to autonomously generatesuch explanations, and verify that they are as effective as its scriptedcounterpart and generalizable across environments.Our work brings XAI techniques into the domain of fault recov-ery and aims to aid non-expert users (1) understand unexpectedfailures of a complex robot system and (2) provide recovery so-lutions in such an event. Although our work includes importantcontributions, there are limitations that should be addressed byfuture work. First, while the context-based-history explanationsare useful for assisting in failure recovery, they are not guaran-teed to be useful to all non-experts. Therefore future work canexplore tailoring explanations to individual users, perhaps withthe reinforcement learning techniques used in recommender sys-tems [45]. Second, our work has characterized the utility of contextand history in providing meaningful E πππ , but we have assumedthat explanations can be arbitrarily long. Future work should in-vestigate additional factors that characterize a good E πππ , and thetradeoffs of providing more information vs. remaining concise. Fi-nally, while the current encoder-decoder model can generalize overvarying failure scenarios, there is still room to improve its gener-alizability to additional situations. Future work can investigate awider range of simulation domains, tasks, and failures. This material is based upon work supported by the NSF GraduateResearch Fellowship under Grant No. DGE-1650044.
EFERENCES [1] Boussad Abci, Maan El Badaoui El Najjar, Vincent Cocquempot, and GΓ©raldDherbomez. 2020. An informational approach for sensor and actuator faultdiagnosis for autonomous mobile robots.
Journal of Intelligent & Robotic Systems
99, 2 (2020), 387β406.[2] Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: Asurvey on Explainable Artificial Intelligence (XAI).
IEEE Access
Proc. of the 17th International Conference on Autonomous Agents andMultiAgent Systems . 1168β1176.[4] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural ma-chine translation by jointly learning to align and translate. In .[5] Siddhartha Banerjee, Angel Daruna, David Kent, Weiyu Liu, Jonathan Balloch,Abhinav Jain, Akshay Krishnan, Muhammad Asif Rana, Harish Ravichandar,Binit Shah, Nithin Shrivatsav, and Sonia Chernova. 2019. Taking Recoveries toTask: Recovery-Driven Development for Recipe-based Robot Tasks.
ISRR (2019).[6] Joost Bastings. 2018. The Annotated Encoder-Decoder with Attention.[7] Andrea Bauer, Dirk Wollherr, and Martin Buss. 2008. Humanβrobot collaboration:a survey.
International Journal of Humanoid Robotics
5, 01 (2008), 47β66.[8] Anders BillesΓΈ Beck, Anders Due Schwartz, Andreas Rune Fugl, Martin Naumann,and BjΓΆrn Kahl. 2015. Skill-based Exception Handling and Error Recovery forCollaborative Industrial Robots.. In
FinE-R@ IROS . 5β10.[9] Richard Bloss. 2011. Mobile hospital robots cure numerous logistic needs.
Indus-trial Robot: An International Journal (2011).[10] Adrian Boteanu, David Kent, Anahita Mohseni-Kabir, Charles Rich, and SoniaChernova. 2015. Towards robot adaptability in new situations. In .[11] Tathagata Chakraborti, Anagha Kulkarni, Sarath Sreedharan, David E Smith,and Subbarao Kambhampati. 2019. Explicability? legibility? predictability? trans-parency? privacy? security? the emerging landscape of interpretable agent behav-ior. In
Proc. of the international conference on automated planning and scheduling ,Vol. 29. 86β96.[12] Tathagata Chakraborti, Sarath Sreedharan, and Subbarao Kambhampati. 2020.The Emerging Landscape of Explainable AI Planning and Decision Making. arXivpreprint arXiv:2002.11697 (2020).[13] Tathagata Chakraborti, Sarath Sreedharan, Yu Zhang, and Subbarao Kambham-pati. 2017. Plan explanations as model reconciliation: Moving beyond explanationas soliloquy. arXiv preprint arXiv:1701.08317 (2017).[14] Kai-Hsiung Chang, Hyungoo Han, and William B Day. 1993. A comparisonof failure-handling approaches for planning systemsβReplanning vs. recovery.
Applied Intelligence
3, 4 (1993), 275β300.[15] Chao Chen, Rui Xu, Shengying Zhu, Zhaoyu Li, and Huiping Jiang. 2020. RPRS:A reactive Plan repair strategy for rapid response to Plan failures of deep spacemissions.
Acta Astronautica (2020).[16] Kyunghyun Cho, Bart Van MerriΓ«nboer, Caglar Gulcehre, Dzmitry Bahdanau,Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phraserepresentations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).[17] D. Crestani, K. Godary-Dejean, and L. Lapierre. 2015. Enhancing fault toleranceof autonomous mobile robots.
Robotics and Autonomous Systems
68 (jun 2015),140β155. https://doi.org/10.1016/j.robot.2014.12.015[18] Devleena Das and Sonia Chernova. 2020. Leveraging rationales to improve humantask performance. In
Proc. of the 25th International Conference on Intelligent UserInterfaces . 510β518.[19] Upol Ehsan, Brent Harrison, Larry Chan, and Mark O Riedl. 2018. Rational-ization: A neural machine translation approach to generating natural languageexplanations. In
Proc. of the 2018 AAAI/ACM Conference on AI, Ethics, and Society .81β87.[20] Upol Ehsan, Pradyumna Tambwekar, Larry Chan, Brent Harrison, and Mark ORiedl. 2019. Automated rationale generation: a technique for explainable AI andits effects on human perceptions. In
Proc. of the 24th International Conference onIntelligent User Interfaces . 263β274.[21] David Gunning and David W Aha. 2019. DARPAβs explainable artificial intelli-gence program.
AI Magazine
40, 2 (2019), 44β58.[22] Martin HΓ€gele, Klas Nilsson, J Norberto Pires, and Rainer Bischoff. 2016. Industrialrobotics. In
Springer handbook of robotics . Springer, 1385β1422.[23] Kristian J Hammond. 1990. Explaining and repairing plans that fail.
Artificialintelligence
45, 1-2 (1990), 173β228.[24] JΓΆrg Hoffmann and Daniele Magazzeni. 2019. Explainable AI Planning (XAIP):Overview and the Case of Contrastive Explanation. In
Reasoning Web. ExplainableArtificial Intelligence . Springer, 277β282.[25] John Hu, Aaron Edsinger, Yi-Je Lim, Nick Donaldson, Mario Solano, AaronSolochek, and Ronald Marchessault. 2011. An advanced medical robotic sys-tem augmenting healthcare capabilities-robotic nursing assistant. In . IEEE, 6264β6269. [26] Subbarao Kambhampati. 2019. Synthesizing explainable behavior for human-AIcollaboration. In
Proc. of the 18th International Conference on Autonomous Agentsand Multi-Agent Systems . 1β2.[27] Gayane Kazhoyan, Arthur Niedzwiecki, and Michael Beetz. 2020. Towards PlanTransformations for Real-World Mobile Fetch and Place. In . IEEE, 11011β11017.[28] Eliahu Khalastchi and Meir Kalech. 2018. A sensor-based approach for faultdetection and diagnosis for robotic systems.
Autonomous Robots
42, 6 (aug 2018),1231β1248. https://doi.org/10.1007/s10514-017-9688-z[29] Eliahu Khalastchi and Meir Kalech. 2018. On fault detection and diagnosis inrobotic systems.
ACM Computing Surveys (CSUR)
51, 1 (2018), 1β24.[30] Dominik Kirchner, Stefan Niemczyk, and Kurt Geihs. 2014. RoSHA: A Multi-robotSelf-healing Architecture. In
RoboCup 2013: Robot World Cup XVII (lecture noed.). Springer, Berlin, Heidelberg, 304β315.[31] Ross A Knepper, Stefanie Tellex, Adrian Li, Nicholas Roy, and Daniela Rus. 2015.Recovering from failure by asking for help.
Autonomous Robots
39, 3 (2015),347β362.[32] Benjamin Krarup, Michael Cashmore, Daniele Magazzeni, and Tim Miller. 2019.Model-based contrastive explanations for explainable planning. (2019).[33] Jim Lawton. 2016. Collaborative robots.
International Society of Automation (2016), 12β14.[34] Lakshmi Nair and Sonia Chernova. 2020. Feature Guided Search for CreativeProblem Solving Through Tool Construction. arXiv preprint arXiv:2008.10685 (2020).[35] Daehyung Park, Hokeun Kim, Yuuna Hoshi, Zackory Erickson, Ariel Kapusta,and Charles C. Kemp. 2017. A multimodal execution monitor with anomalyclassification for robot-assisted feeding. In
IROS . IEEE, 5406β5413.[36] Lynne Parker and Balajee Kannan. 2006. Adaptive Causal Models for FaultDiagnosis and Recovery in Multi-Robot Teams. In
IROS . IEEE, 2703β2710.[37] Ola Pettersson, L. Karlsson, and A. Saffiotti. 2007. Model-Free Execution Mon-itoring in Behavior-Based Robotics.
IEEE Transactions on Systems, Man andCybernetics, Part B (Cybernetics)
37, 4 (aug 2007), 890β901. https://doi.org/10.1109/TSMCB.2007.895359[38] Arun Rai. 2020. Explainable AI: From black box to glass box.
Journal of theAcademy of Marketing Science
48, 1 (2020), 137β141.[39] Vasumathi Raman and Hadas Kress-Gazit. 2012. Explaining impossible high-levelrobot behaviors.
IEEE Transactions on Robotics
29, 1 (2012), 94β104.[40] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why shouldI trust you?" Explaining the predictions of any classifier. In
Proc. of the 22ndACM SIGKDD international conference on knowledge discovery and data mining .1135β1144.[41] Allison SauppΓ© and Bilge Mutlu. 2015. The social impact of a robot co-worker inindustrial settings. In
Proc. of the 33rd annual ACM conference on human factorsin computing systems . 3613β3622.[42] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan-tam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations fromdeep networks via gradient-based localization. In
Proc. of the IEEE internationalconference on computer vision . 618β626.[43] Sarath Sreedharan, Siddharth Srivastava, David E Smith, and Subbarao Kambham-pati. 2019. Why Canβt You Do That HAL? Explaining Unsolvability of PlanningTasks. In
IJCAI . 1422β1430.[44] V. Verma, G. Gordon, R. Simmons, and S. Thrun. 2004. Real-time fault diagnosis.
IEEE Robotics & Automation Magazine
11, 2 (jun 2004), 56β66. https://doi.org/10.1109/MRA.2004.1310942[45] Xiting Wang, Yiru Chen, Jie Yang, Le Wu, Zhengtao Wu, and Xing Xie. 2018. Areinforcement learning framework for explainable recommendation. In . IEEE, 587β596.[46] Melonee Wise, Michael Ferguson, Derek King, Eric Diehr, and David Dymesich.2016. Fetch and freight: Standard platforms for service robot applications. In
Workshop on autonomous mobile service robots .[47] Hongmin Wu, Shuangqi Luo, Longxin Chen, Shuangda Duan, SakmongkonChumkamon, Dong Liu, Yisheng Guan, and Juan Rojas. 2018. Endowing Robotswith Longer-term Autonomy by Recovering from External Disturbances in Ma-nipulation through Grounded Anomaly Classification and Recovery Policies. (sep2018). arXiv:1809.03979[48] Mike Wu, Michael C Hughes, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, andFinale Doshi-Velez. 2017. Beyond sparsity: Tree regularization of deep modelsfor interpretability. arXiv preprint arXiv:1711.06178 (2017).[49] Pin-Chu Yang, Kazuma Sasaki, Kanata Suzuki, Kei Kase, Shigeki Sugano, andTetsuya Ogata. 2016. Repeatable folding task by humanoid robot worker usingdeep learning.
IEEE Robotics and Automation Letters
2, 2 (2016), 397β403.[50] Safdar Zaman, Gerald Steinbauer, Johannes Maurer, Peter Lepej, and SuzanaUran. 2013. An integrated model-based diagnosis and repair architecture forROS-based robot systems. In . IEEE, 482β489. https://doi.org/10.1109/ICRA.2013.6630618[51] Quanshi Zhang, Yu Yang, Haotian Ma, and Ying Nian Wu. 2019. Interpretingcnns via decision trees. In
Proc. of the IEEE Conference on Computer Vision andPattern Recognition . 6261β6270.52] Yu Zhang, Sarath Sreedharan, Anagha Kulkarni, Tathagata Chakraborti,Hankz Hankui Zhuo, and Subbarao Kambhampati. 2017. Plan explicability andpredictability for robot task planning. In . IEEE, 1313β1320. [53] Zhengjiang Zhang and Junghui Chen. 2019. Fault detection and diagnosis basedon particle filters combined with interactive multiple-model estimation in dy-namic process systems.