Revisiting Attention Weights as Interpretations of Message-Passing Neural Networks
CoRR(2024)
摘要
The self-attention mechanism has been adopted in several widely-used
message-passing neural networks (MPNNs) (e.g., GATs), which adaptively controls
the amount of information that flows along the edges of the underlying graph.
This usage of attention has made such models a baseline for studies on
explainable AI (XAI) since interpretations via attention have been popularized
in various domains (e.g., natural language processing and computer vision).
However, existing studies often use naive calculations to derive attribution
scores from attention, and do not take the precise and careful calculation of
edge attribution into consideration. In our study, we aim to fill the gap
between the widespread usage of attention-enabled MPNNs and their potential in
largely under-explored explainability, a topic that has been actively
investigated in other areas. To this end, as the first attempt, we formalize
the problem of edge attribution from attention weights in GNNs. Then, we
propose GATT, an edge attribution calculation method built upon the computation
tree. Through comprehensive experiments, we demonstrate the effectiveness of
our proposed method when evaluating attributions from GATs. Conversely, we
empirically validate that simply averaging attention weights over graph
attention layers is insufficient to interpret the GAT model's behavior. Code is
publicly available at https://github.com/jordan7186/GAtt/tree/main.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要