Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis
CoRR(2023)
摘要
After a large language model (LLM) is deployed on edge devices, it is
desirable for these devices to learn from user-generated conversation data to
generate user-specific and personalized responses in real-time. However,
user-generated data usually contains sensitive and private information, and
uploading such data to the cloud for annotation is not preferred if not
prohibited. While it is possible to obtain annotation locally by directly
asking users to provide preferred responses, such annotations have to be sparse
to not affect user experience. In addition, the storage of edge devices is
usually too limited to enable large-scale fine-tuning with full user-generated
data. It remains an open question how to enable on-device LLM personalization,
considering sparse annotation and limited on-device storage. In this paper, we
propose a novel framework to select and store the most representative data
online in a self-supervised way. Such data has a small memory footprint and
allows infrequent requests of user annotations for further fine-tuning. To
enhance fine-tuning quality, multiple semantically similar pairs of question
texts and expected responses are generated using the LLM. Our experiments show
that the proposed framework achieves the best user-specific content-generating
capability (accuracy) and fine-tuning speed (performance) compared with vanilla
baselines. To the best of our knowledge, this is the very first on-device LLM
personalization framework.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要