AI Research

Explaining the Paper: Hopfield Networks is All You Need

Yannic Kilcher explains the paper “Hopfield Networks is All You Need.” Hopfield Networks are one of the classic models of biological memory networks. This paper generalizes modern Hopfield Networks to continuous states and shows that the corresponding update rule is equal to the attention mechanism used in modern Transformers. It further analyzes a pre-trained BERT […]

Read More
Neural Networks

Neural Networks and Linear Regression

On Friday, someone asked me about linear regression with neural networks.  I didn’t have a good answer – I knew that you *could* do linear regression but neural networks, but never had actually done it in practice. Promising to learn more, I came across this video by giant_neural_network on YouTube.

Read More