Prompts as Auto-Optimized Training Hyperparameters: Training Best-in-Class IR Models from Scratch with 10 Gold Labels
CoRR(2024)
摘要
We develop a method for training small-scale (under 100M parameter) neural
information retrieval models with as few as 10 gold relevance labels. The
method depends on generating synthetic queries for documents using a language
model (LM), and the key step is that we automatically optimize the LM prompt
that is used to generate these queries based on training quality. In
experiments with the BIRCO benchmark, we find that models trained with our
method outperform RankZephyr and are competitive with RankLLama, both of which
are 7B parameter models trained on over 100K labels. These findings point to
the power of automatic prompt optimization for synthetic dataset generation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要