Comparing Hallucination Detection Metrics for Multilingual Generation
CoRR(2024)
摘要
While many automatic hallucination detection techniques have been proposed
for English texts, their effectiveness in multilingual contexts remains
unexplored. This paper aims to bridge the gap in understanding how these
hallucination detection metrics perform on non-English languages. We evaluate
the efficacy of various detection metrics, including lexical metrics like ROUGE
and Named Entity Overlap and Natural Language Inference (NLI)-based metrics, at
detecting hallucinations in biographical summaries in many languages; we also
evaluate how correlated these different metrics are to gauge whether they
measure the same phenomena. Our empirical analysis reveals that while lexical
metrics show limited effectiveness, NLI-based metrics perform well in
high-resource languages at the sentence level. In contrast, NLI-based metrics
often fail to detect atomic fact hallucinations. Our findings highlight
existing gaps in multilingual hallucination detection and motivate future
research to develop more robust detection methods for LLM hallucination in
other languages.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要