AI
Research
Explaining the Paper: Hopfield Networks is All You Need
-
Frank
- August 10, 2020
- AI
- artificial intelligence
- Arxiv
- attention
- attention is all you need
- Bert
- binary
- continuous
- Deep Learning
- energy function
- error
- explained
- exponental
- gru
- hochreiter
- hopfield
- hopfield network
- key
- lse
- LSTM
- Machine Learning
- metastable
- Neural Networks
- Paper
- pattern
- query
- retrieval
- RNN
- routing
- Schmidhuber
- separation
- store
- transformer
- update rule
- value
Yannic Kilcher explains the paper “Hopfield Networks is All You Need.” Hopfield Networks are one of the classic models of biological memory networks. This paper generalizes modern Hopfield Networks to continuous states and shows that the corresponding update rule is equal to the attention mechanism used in modern Transformers. It further analyzes a pre-trained BERT […]
Read More