Causal Explanations for Sequential Decision-Making in Multi-Agent Systems
CoRR(2023)
Abstract
We present CEMA: Causal Explanations in Multi-Agent systems; a framework for
creating causal natural language explanations of an agent's decisions in
dynamic sequential multi-agent systems to build more trustworthy autonomous
agents. Unlike prior work that assumes a fixed causal structure, CEMA only
requires a probabilistic model for forward-simulating the state of the system.
Using such a model, CEMA simulates counterfactual worlds that identify the
salient causes behind the agent's decisions. We evaluate CEMA on the task of
motion planning for autonomous driving and test it in diverse simulated
scenarios. We show that CEMA correctly and robustly identifies the causes
behind the agent's decisions, even when a large number of other agents is
present, and show via a user study that CEMA's explanations have a positive
effect on participants' trust in autonomous vehicles and are rated as high as
high-quality baseline explanations elicited from other participants. We release
the collected explanations with annotations as the HEADD dataset.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined