Paper

AI Natural Language Processing

ROME: Locating and Editing Factual Associations in GPT (Paper Explained & Author Interview)

Large Language Models have the ability to store vast amounts of facts about the world. But little is known, how these models actually do this. This paper aims at discovering the mechanism and location of storage and recall of factual associations in GPT models, and then proposes a mechanism for the targeted editing of such […]

Read More
AI Mathematics

This is a game changer! (AlphaTensor by DeepMind explained)

Matrix multiplication is the most used mathematical operation in all of science and engineering. Speeding this up has massive consequences. Thus, over the years, this operation has become more and more optimized. A fascinating discovery was made when it was shown that one actually needs less than N^3 multiplication operations to multiply to NxN matrices. […]

Read More
AI Hardware Research

How to make your CPU as fast as a GPU – Advances in Sparsity w/ Nir Shavit

Sparsity is awesome, but only recently has it become possible to properly handle sparse models at good performance. Neural Magic does exactly this, using a plain CPU. No specialized hardware needed, just clever algorithms for pruning and forward-propagation of neural networks. Nir Shavit and I talk about how this is possible, what it means in […]

Read More
Generative AI

[ML News] Stable Diffusion Takes Over! (Open Source AI Art)

Stable Diffusion has been released and is riding a wave of creativity and collaboration. But not everyone is happy about this — especially artists!

Read More
AI Generative AI

Parti – Scaling Autoregressive Models for Content-Rich Text-to-Image Generation

Parti is a new autoregressive text-to-image model that shows just how much scale can achieve. This model’s outputs are crips, accurate, realistic, and can combine arbitrary styles, concepts, and fulfil even challenging requests. Yannic explains the research paper. Time stamps: 0:00 – Introduction 2:40 – Example Outputs 6:00 – Model Architecture 17:15 – Datasets (incl. […]

Read More
AI

Is This the Worst AI Ever?

GPT-4chan was trained on over 3 years of posts from 4chan’s “politically incorrect” (/pol/) board. (and no, this is not GPT-4) You can imagine what it learned. Maybe we need to be better people so that we can make sure our AI overlords will have better behavior to model.

Read More
AI Generative AI

The Weird and Wonderful World of AI Art (w/ Author Jack Morris)

Since the release of CLIP, the world of AI art has seen an unprecedented level of acceleration in what’s possible to do. Whereas image generation had previously been mostly in the domain of scientists, now a community of professional artists, researchers, and amateurs are sending around colab notebooks and sharing their creations via social media. […]

Read More
AI Computer Vision Open Source

Exploring the LAION-5B: a 5 billion image-text-pairs dataset

LAION-5B is an open, free dataset consisting of over 5 billion image-text-pairs. Today’s video is an interview with three of its creators. We dive into the mechanics and challenges of operating at such large scale, how to keep cost low, what new possibilities are enabled with open datasets like this, and how to best handle […]

Read More