Sparse and Structured Hopfield Networks
CoRR(2024)
摘要
Modern Hopfield networks have enjoyed recent interest due to their connection
to attention in transformers. Our paper provides a unified framework for sparse
Hopfield networks by establishing a link with Fenchel-Young losses. The result
is a new family of Hopfield-Fenchel-Young energies whose update rules are
end-to-end differentiable sparse transformations. We reveal a connection
between loss margins, sparsity, and exact memory retrieval. We further extend
this framework to structured Hopfield networks via the SparseMAP
transformation, which can retrieve pattern associations instead of a single
pattern. Experiments on multiple instance learning and text rationalization
demonstrate the usefulness of our approach.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要