Consecutive Model Editing with Batch alongside HooK Layers
arxiv(2024)
摘要
As the typical retraining paradigm is unacceptably time- and
resource-consuming, researchers are turning to model editing in order to seek
an effective, consecutive, and batch-supportive way to edit the model behavior
directly. Despite all these practical expectations, existing model editing
methods fail to realize all of them. Furthermore, the memory demands for such
succession-supportive model editing approaches tend to be prohibitive,
frequently necessitating an external memory that grows incrementally over time.
To cope with these challenges, we propose COMEBA-HK, a model editing method
that is both consecutive and batch-supportive. COMEBA-HK is memory-friendly as
it only needs a small amount of it to store several hook layers with updated
weights. Experimental results demonstrate the superiority of our method over
other batch-supportive model editing methods under both single-round and
consecutive batch editing scenarios. Extensive analyses of COMEBA-HK have been
conducted to verify the stability of our method over 1) the number of
consecutive steps and 2) the number of editing instance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要