Half Ideas – Startups and Entrepreneurship takes a closer look at GPT-3 and what it means for AI.
GPT 3 can write poetry, translate text, chat convincingly, and answer abstract questions. It’s being used to code, design and much more. I’ll give you a demo of some of the latest in this technology and some of how it works.
GPT 3 has been developed for a number of years. One of the early papers published was on Generative Pre-Training. The idea behind generative pre-training (GPT) is that while most AI’s are trained on labeled data, there’s a ton of data that isn’t labeled. If you can evaluate the words and use them to train and tune the AI it can start to create predictions of future text on the unlabeled data. You repeat the process until predictions start to converge.