Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
CoRR(2024)
摘要
Interpretability research aims to bridge the gap between the empirical
success and our scientific understanding of the inner workings of large
language models (LLMs). However, most existing research in this area focused on
analyzing a single mechanism, such as how models copy or recall factual
knowledge. In this work, we propose the formulation of competition of
mechanisms, which instead of individual mechanisms focuses on the interplay of
multiple mechanisms, and traces how one of them becomes dominant in the final
prediction. We uncover how and where the competition of mechanisms happens
within LLMs using two interpretability methods, logit inspection and attention
modification. Our findings show traces of the mechanisms and their competition
across various model components, and reveal attention positions that
effectively control the strength of certain mechanisms. Our code and data are
at https://github.com/francescortu/Competition_of_Mechanisms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要