Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions
-
Frank
- November 28, 2021
- AI
- artificial intelligence
- Arxiv
- backpropagation through algorithms
- Deep Learning
- deep learning discrete
- discrete backpropagation
- discrete deep learning
- explained
- gradient discrete
- gradient of an algorithm
- imle
- implicit mle
- Machine Learning
- maximum likelihood
- Neural Networks
- Paper
Backpropagation is the workhorse of deep learning, but unfortunately, it only works for continuous functions that are amenable to the chain rule of differentiation. Since discrete algorithms have no continuous derivative, deep networks with such algorithms as part of them cannot be effectively trained using backpropagation. This paper presents a method to incorporate a large […]
Read More
[ML News] Google introduces Pathways | OpenAI solves Math Problems | Meta goes First Person
-
Frank
- November 6, 2021
- AI
- artificial intelligence
- Arxiv
- bigscience
- bigsciencew
- dataset viewer
- Deep Learning
- DeepMind
- deepmind lecture series
- ego4d
- explained
- Google AI
- google pathways
- huggingface
- huggingface hub
- Jeff Dean
- language model math
- Machine Learning
- machine learning news
- meta
- meta ai
- Neural Networks
- OpenAI
- openai math
- Paper
- pathways model
- sam altman
- sparse neural network
- t0
- tech news
- tzero
Yannic provides us with a dose of Machine Learning News. Time stamps: 0:00 – Intro 0:20 – Sponsor: Weights & Biases 2:10 – Google Introduces Pathways AI Architecture 6:30 – OpenAI trains Language Models to do High School Math 8:25 – Sam Altman says Neural Networks truly learn 9:35 – Google AI researchers frustrated with […]
Read More
[ML News] New ImageNet SOTA | Uber’s H3 hexagonal coordinate system | New text-image-pair dataset
Yannic provides the latest news in machine learning in this video. Time Stamps: 0:00 – Intro 0:20 – TruthfulQA benchmark shines new light on GPT-3 2:00 – LAION-400M image-text-pair dataset 4:10 – GoogleAI’s EfficientNetV2 and CoAtNet 6:15 – Uber’s H3: A hexagonal coordinate system 7:40 – AWS NeurIPS 2021 DeepRacer Challenge 8:15 – Helpful Libraries […]
Read More
PonderNet: Learning to Ponder
-
Frank
- August 24, 2021
- AI
- Andrea Banino
- artificial intelligence
- Arxiv
- broader impact
- Charles Blundell
- Deep Learning
- deep learning classic algorithms
- deep learning recurrent computation
- deep network learning to stop
- DeepMind
- deepmind pondernet
- dynamic computation
- dynamic recurrent network
- explained
- halting probability
- Jan Balaguer
- Machine Learning
- Neural Networks
- Paper
- pondernet
- pondernet explained
- pondernet learning to ponder
Humans don’t spend the same amount of mental effort on all problems equally. Instead, we respond quickly to easy tasks, and we take our time to deliberate hard tasks. DeepMind’s PonderNet attempts to achieve the same by dynamically deciding how many computation steps to allocate to any single input sample. This is done via a […]
Read More
Why AI is Harder Than We Think
-
Frank
- May 3, 2021
- AI
- ai spring
- AI Winter
- alan turing
- are neural networks artificial intelligence
- artificial intelligence
- Arxiv
- can machines be conscious
- can machines think
- Deep Learning
- deep learning artificial intelligence
- deep learning tutorial
- elon musk artificial intelligence
- expert systems
- explained
- introduction to deep learning
- Machine Learning
- marvin minsky
- Neural Networks
- Paper
- Self Driving Cars
- what is deep learning
- why is ai hard
- why is deep learning important
Yannic Kilcher explains how the AI community has gone through regular cycles of AI Springs, where rapid progress gave rise to massive overconfidence, high funding, and overpromise, followed by these promises being unfulfilled, subsequently diving into periods of disenfranchisement and underfunding, called AI Winters. This video he explores a paper which examines the reasons for […]
Read More
GLOM: How to represent part-whole hierarchies in a neural network (Geoff Hinton’s Paper Explained)
-
Frank
- March 3, 2021
- AI
- artificial intelligence
- Arxiv
- attention mechanism
- Capsule Networks
- capsule networks explained
- column
- Computer Vision
- consensus algorithm
- Deep Learning
- deep learning tutorial
- explained
- Geoff Hinton
- geoff hinton capsule networks
- geoff hinton neural networks
- Geoffrey Hinton
- geoffrey hinton deep learning
- geoffrey hinton glom
- glom model
- Google AI
- Google Brain
- hinton glom
- introduction to deep learning
- Machine Learning
- Neural Networks
- Schmidhuber
- transformer
Yannic Kilcher covers a paper where Geoffrey Hinton describes GLOM, a Computer Vision model that combines transformers, neural fields, contrastive learning, capsule networks, denoising autoencoders and RNNs. GLOM decomposes an image into a parse tree of objects and their parts. However, unlike previous systems, the parse tree is constructed dynamically and differently for each input, […]
Read More){kind=link}
TransGAN: Two Transformers Can Make One Strong GAN
-
Frank
- February 19, 2021
- AI
- artificial intelligence
- Arxiv
- attention is all you need
- attention mechanism
- attention neural networks
- Deep Learning
- deep learning explained
- generative adversarial network
- local attention
- Machine Learning
- machine learning explained
- multihead attention
- Neural Networks
- paper explained
- pixelshuffle
- self attention
- superresolution
- transformer gan
- transformer gans
- transformer generative adversarial network
- transformer generator
- transgan
- vision transformer
Generative Adversarial Networks (GANs) hold the state-of-the-art when it comes to image generation. However, while the rest of computer vision is slowly taken over by transformers or other attention-based architectures, all working GANs to date contain some form of convolutional layers. This paper changes that and builds TransGAN, the first GAN where both the generator […]
Read More
Deep Networks Are Kernel Machines
-
Frank
- February 18, 2021
- AI
- artificial intelligence
- Arxiv
- data representations
- Deep Learning
- deep neural networks
- explained
- kernel machines
- kernel trick
- learning theory
- Linear Regression
- Machine Learning
- machine learning theory
- math proof
- nearest neighbor
- Neural Networks
- neural networks gradient descent
- Paper
- pedro domingos
- proof
- representation learning
- representations
- representer theorem
- SGD
- stochastic gradient descent
- Support Vector Machine
- SVM
- what is deep learning
Yannic Kilcher explains the paper “Every Model Learned by Gradient Descent Is Approximately a Kernel Machine.” Deep Neural Networks are often said to discover useful representations of the data. However, this paper challenges this prevailing view and suggest that rather than representing the data, deep neural networks store superpositions of the training data in their […]
Read More
NFNets: High-Performance Large-Scale Image Recognition Without Normalization
-
Frank
- February 15, 2021
- AI
- artificial intelligence
- Arxiv
- batch norm
- batch normalization
- batchnorm
- best imagenet model
- best neural network
- Deep Learning
- deep learning code
- deep learning tutorial
- Deep Mind
- DeepMind
- distributed training
- explained
- gradient clipping
- imagenet
- JAX
- layer normalization
- Machine Learning
- machine learning explained
- machine learning tutorial
- mean shift
- ml code
- Neural Networks
- nfnet
- nfnets
- nfnets code
- nfresnet
- normalizer-free
- Paper
- weight standardization
Yannic Kilcher explains NFNets in this video.
Read More
SingularityNET – A Decentralized, Open Market and Network for AIs
-
Frank
- February 11, 2021
- AGI
- agi token
- AI
- API
- Artificial General Intelligence
- artificial intelligence
- Arxiv
- Ben Goertzel
- benefit
- Blockchain
- cardano
- Deep Learning
- deoldify
- erc20
- Ethereum
- explained
- goertzel
- Governance
- Graph
- hanson
- hanson robotics
- ICO
- inflation
- intelligence
- liquid rank
- liquidrank
- Machine Learning
- matchmaking
- network
- Neural Networks
- offer coin
- offer networks
- offercoin
- offernetworks
- open cog
- opencog
- Paper
- reputation
- reputation system
- Singularity
- singularitynet
- sophia
- tontoni phi
Yannic Kilcher explains this white paper on SingularityNET. Big Tech is currently dominating the pursuit of ever more capable AI. This happens behind closed doors and results in a monopoly of power. SingularityNET is an open, decentralized network where anyone can offer and consume AI services, and where AI agents can interlink with each other […]
Read More