AI Language Models & Transformers
Plausible text generation has been around for a couple of years, but how does it work – and what’s next? Rob Miles on Language Models and Transformers.
Read More
GPT-3: Language Models are Few-Shot Learners
-
Frank
- June 2, 2020
- AI
- artificial intelligence
- Arxiv
- attention
- autoregressive
- Bert
- boolq
- common crawl
- context
- corpus
- deep language
- Deep Learning
- explained
- Few Shot
- glue
- GPT-2
- gpt-3
- gpt2
- gpt3
- heads
- language model
- Machine Learning
- Math
- Microsoft
- mlm
- Natural Language Processing
- natural questions
- Neural Networks
- news
- NLP
- OpenAI
- Paper
- preplexity
- question answering
- sota
- strings
- superglue
- training data
- Transformers
- turing
- Wikipedia
- zero shot
How far can you go with ONLY language modeling? Can a large enough language model perform NLP task out of the box? OpenAI take on these and other questions by training a transformer that is an order of magnitude larger than anything that has ever been built before and the results are astounding. Yannic Kilcher […]
Read More
OpenAI Model Generates Python Code
-
Frank
- June 1, 2020
- AI
- arguments
- artificial intelligence
- Arxiv
- Build
- Code
- code completion
- completion
- Dataset
- Deep Learning
- docstring
- explained
- function body
- generate
- GitHub
- gpt2
- Intellicode
- intellisense
- interpolate
- language model
- Machine Learning
- Microsoft
- msbuild
- name
- Neural Networks
- OpenAI
- Paper
- programmer
- Python
- smart
- stackoverflow
- vscode
OpenAI demonstrated what happens when you learn a language model on thousands of GitHub Python repositories Yannic Kilcher talks about this code completion engine which can write an entire function from just the name. Source Clip: https://youtu.be/fZSFNUT6iY8 Full Video: https://www.pscp.tv/Microsoft/1OyKAYWPRrWKb
Read More