Deep Unlearning: Fast and Efficient Training-free Approach to Class Forgetting
arxiv(2023)
摘要
Machine unlearning is a prominent and challenging field, driven by regulatory
demands for user data deletion and heightened privacy awareness. Existing
approaches involve retraining model or multiple finetuning steps for each
deletion request, often constrained by computational limits and restricted data
access. In this work, we introduce a novel class unlearning algorithm designed
to strategically eliminate specific classes from the learned model. Our
algorithm first estimates the Retain and the Forget Spaces using Singular Value
Decomposition on the layerwise activations for a small subset of samples from
the retain and unlearn classes, respectively. We then compute the shared
information between these spaces and remove it from the forget space to isolate
class-discriminatory feature space. Finally, we obtain the unlearned model by
updating the weights to suppress the class discriminatory features from the
activation spaces. We demonstrate our algorithm's efficacy on ImageNet using a
Vision Transformer with only ∼ 1.5% drop in retain accuracy compared to
the original model while maintaining under 1% accuracy on the unlearned
class samples. Further, our algorithm consistently performs well when subject
to Membership Inference Attacks showing 7.8% improvement on average across a
variety of image classification datasets and network architectures, as compared
to other baselines while being ∼ 6 × more computationally efficient.
Our code is available at https://github.com/sangamesh-kodge/class_forgetting.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要