AlphaCode Explained: AI Code Generation
-
Frank
- February 15, 2022
- AI
- AI AlphaCode
- AI Codex
- Alpha Code
- AlphaCode
- AlphaCode AI
- artificial intelligence
- attention
- billion
- billion parameter
- codex
- codex ai
- Deep Mind
- DeepMind
- explained
- Explanation
- gopher explained
- Gopher Model
- GPT-2
- gpt-3
- gpt-4
- language modeling
- Large Language Model
- logic reasoning
- Machine Learning
- ML
- Natural Language Processing
- NLP
- OpenAI
- OpenAI Codex
- Retro
- self attention
- Self-Attention
- Self-Supervised Learning
- sota
- State of the Art
- Text Generation
- transformer
AlphaCode is DeepMind’s new massive language model for generating code. It is similar to OpenAI Codex, except for in the paper they provide a bit more analysis. The field of NLP within AI and ML has exploded get a lot more papers all the time. Hopefully this video can help you understand how AlphaCode works […]
Read More
Explaining the Paper: Hopfield Networks is All You Need
-
Frank
- August 10, 2020
- AI
- artificial intelligence
- Arxiv
- attention
- attention is all you need
- Bert
- binary
- continuous
- Deep Learning
- energy function
- error
- explained
- exponental
- gru
- hochreiter
- hopfield
- hopfield network
- key
- lse
- LSTM
- Machine Learning
- metastable
- Neural Networks
- Paper
- pattern
- query
- retrieval
- RNN
- routing
- Schmidhuber
- separation
- store
- transformer
- update rule
- value
Yannic Kilcher explains the paper “Hopfield Networks is All You Need.” Hopfield Networks are one of the classic models of biological memory networks. This paper generalizes modern Hopfield Networks to continuous states and shows that the corresponding update rule is equal to the attention mechanism used in modern Transformers. It further analyzes a pre-trained BERT […]
Read More
Object-Centric Learning with Slot Attention
-
Frank
- July 1, 2020
- AI
- artificial intelligence
- Arxiv
- attention
- attention mechanism
- capsules
- clevr
- CNN
- Convolutional Neural Network
- Deep Learning
- detr
- disentanglement
- embeddings
- encoder
- ethz
- explained
- gru
- LSTM
- Machine Learning
- Neural Networks
- objects
- Paper
- permutation invariant
- render
- routing
- set
- slots
- tetris
- transformer
- vision
- weight sharing
Visual scenes are often comprised of sets of independent objects. Yet, current vision models make no assumptions about the nature of the pictures they look at. Yannic Kilcher explore a paper on object-centric learning. By imposing an objectness prior, this paper a module that is able to recognize permutation-invariant sets of objects from pixels in […]
Read More
GPT-3: Language Models are Few-Shot Learners
-
Frank
- June 2, 2020
- AI
- artificial intelligence
- Arxiv
- attention
- autoregressive
- Bert
- boolq
- common crawl
- context
- corpus
- deep language
- Deep Learning
- explained
- Few Shot
- glue
- GPT-2
- gpt-3
- gpt2
- gpt3
- heads
- language model
- Machine Learning
- Math
- Microsoft
- mlm
- Natural Language Processing
- natural questions
- Neural Networks
- news
- NLP
- OpenAI
- Paper
- preplexity
- question answering
- sota
- strings
- superglue
- training data
- Transformers
- turing
- Wikipedia
- zero shot
How far can you go with ONLY language modeling? Can a large enough language model perform NLP task out of the box? OpenAI take on these and other questions by training a transformer that is an order of magnitude larger than anything that has ever been built before and the results are astounding. Yannic Kilcher […]
Read More
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
-
Frank
- October 7, 2019
- attention
- attention layers
- Bert
- bidirectional
- Deep Learning
- elmo
- embeddings
- fine tuning
- Jacob Devlin
- Kenton Lee
- Kristina Toutanova
- language
- language modeling
- Machine Learning
- Ming-Wei Chang
- Natural Language Processing
- NLP
- pretrained
- squad
- transformer
- unsupervised
- word vectors
- wordpiece
- Yannic Kilcher
Yannic Kilcher investigates BERT and the white paper associated with it https://arxiv.org/abs/1810.04805 Abstract:We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers. As […]
Read More