Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging
arxiv(2024)
摘要
Model merging is a promising lightweight model empowerment technique that
does not rely on expensive computing devices (e.g., GPUs) or require the
collection of specific training data. Instead, it involves editing different
upstream model parameters to absorb their downstream task capabilities.
However, uncertified model merging can infringe upon the Intellectual Property
(IP) rights of the original upstream models. In this paper, we conduct the
first study on the robustness of IP protection methods in model merging
scenarios. We investigate two state-of-the-art IP protection techniques:
Quantization Watermarking and Instructional Fingerprint, along with various
advanced model merging technologies, such as Task Arithmetic, TIES-MERGING, and
so on. Experimental results indicate that current Large Language Model (LLM)
watermarking techniques cannot survive in the merged models, whereas model
fingerprinting techniques can. Our research aims to highlight that model
merging should be an indispensable consideration in the robustness assessment
of model IP protection techniques, thereby promoting the healthy development of
the open-source LLM community.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要