Do LLMs Find Human Answers To Fact-Driven Questions Perplexing? A Case Study on Reddit
arxiv(2024)
摘要
Large language models (LLMs) have been shown to be proficient in correctly
answering questions in the context of online discourse. However, the study of
using LLMs to model human-like answers to fact-driven social media questions is
still under-explored. In this work, we investigate how LLMs model the wide
variety of human answers to fact-driven questions posed on several
topic-specific Reddit communities, or subreddits. We collect and release a
dataset of 409 fact-driven questions and 7,534 diverse, human-rated answers
from 15 r/AskTopic communities across 3 categories: profession, social
identity, and geographic location. We find that LLMs are considerably better at
modeling highly-rated human answers to such questions, as opposed to
poorly-rated human answers. We present several directions for future research
based on our initial findings.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要