Juru: Legal Brazilian Large Language Model from Reputable Sources
CoRR(2024)
摘要
The high computational cost associated with pretraining large language models
limits their research. Two strategies have emerged to address this issue:
domain specialization and pretraining with high-quality data. To explore these
strategies, we specialized the Sabiá-2 Small model with 1.9 billion unique
tokens from reputable Brazilian legal sources and conducted few-shot
evaluations on legal and general knowledge exams. Our model, Juru, demonstrates
the benefits of domain specialization with a reduced amount of pretraining
data. However, this specialization comes at the expense of degrading
performance in other knowledge areas within the same language. This study
contributes to the growing body of scientific evidence showing that pretraining
data selection may enhance the performance of large language models, enabling
the exploration of these models at a lower cost.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要