Experimental Design for Active Transductive Inference in Large Language Models
arxiv(2024)
摘要
Transduction, the ability to include query-specific examples in the prompt at
inference time, is one of the emergent abilities of large language models
(LLMs). In this work, we propose a framework for adaptive prompt design called
active transductive inference (ATI). We design the LLM prompt by adaptively
choosing few-shot examples for a given inference query. The examples are
initially unlabeled and we query the user to label the most informative ones,
which maximally reduces the uncertainty in the LLM prediction. We propose two
algorithms, GO and SAL, which differ in how the few-shot examples are chosen.
We analyze these algorithms in linear models: first GO and then use its
equivalence with SAL. We experiment with many different tasks and show that GO
and SAL outperform other methods for choosing few-shot examples in the LLM
prompt at inference time.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要