AI Explained covers the latest LLM innovation: Stanford’s Alpaca,
8 years of cost reduction in 5 weeks: how Stanford’s Alpaca model changes everything, including the economics of OpenAI and GPT 4. The breakthrough, using self-instruct, has big implications for Apple’s secret large language model, Baidu’s ErnieBot, Amazon’s attempts and even governmental efforts, like the newly announced BritGPT.
I will go through how Stanford put the model together, why it costs so little, and demonstrate in action versus Chatgpt and GPT 4. And what are the implications of short-circuiting human annotation like this? With analysis of a tweet by Eliezer Yudkowsky, I delve into the workings of the model and the questions it rises.
Links:
- Alpaca: https://crfm.stanford.edu/2023/03/13/alpaca.html
- Ark Forecast: https://research.ark-invest.com/hubfs/1_Download_Files_ARK-Invest/Big_Ideas/ARK%20Invest_013123_Presentation_Big%20Ideas%202023_Final.pdf
- Eliezer Tweet: https://twitter.com/ESYudkowsky/status/1635577836525469697
https://twitter.com/ESYudkowsky/status/1635667349792780288 - Self-Instruct: https://arxiv.org/pdf/2212.10560.pdf
- InstructGPT: https://openai.com/research/instruction-following
- OpenAI Terms: https://openai.com/policies/terms-of-use
- MMLU Test: https://arxiv.org/pdf/2009.03300.pdf
- Apple LLM: https://www.nytimes.com/2023/03/15/technology/siri-alexa-google-assistant-artificial-intelligence.html
- GPT 4 API: https://openai.com/pricing
- Llama Models: https://arxiv.org/pdf/2302.13971.pdf
- BritGPT: https://www.theguardian.com/technology/2023/mar/15/uk-to-invest-900m-in-supercomputer-in-bid-to-build-own-britgpt
- Amazon: https://www.businessinsider.com/amazons-ceo-andy-jassy-on-chat-cpt-ai-2023-2?r=US&IR=T
- AlexaTM: https://arxiv.org/pdf/2208.01448.pdf
- Baidu Ernie: https://www.nytimes.com/2023/03/16/world/asia/china-baidu-chatgpt-ernie.html
- PaLM API: https://developers.googleblog.com/2023/03/announcing-palm-api-and-makersuite.html
){kind=link}