EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluation
arxiv(2024)
摘要
Large language models (LLMs) have gained popularity recently due to their
outstanding performance in various downstream Natural Language Processing (NLP)
tasks. However, low-resource languages are still lagging behind current
state-of-the-art (SOTA) developments in the field of NLP due to insufficient
resources to train LLMs. Ethiopian languages exhibit remarkable linguistic
diversity, encompassing a wide array of scripts, and are imbued with profound
religious and cultural significance. This paper introduces EthioLLM –
multilingual large language models for five Ethiopian languages (Amharic,
Ge'ez, Afan Oromo, Somali, and Tigrinya) and English, and Ethiobenchmark – a
new benchmark dataset for various downstream NLP tasks. We evaluate the
performance of these models across five downstream NLP tasks. We open-source
our multilingual language models, new benchmark datasets for various downstream
tasks, and task-specific fine-tuned language models and discuss the performance
of the models. Our dataset and models are available at the
https://huggingface.co/EthioNLP repository.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要