The latest batch of language models can be much smaller yet achieve GPT-3 like performance by being able to query a database or search the web for information. A key indication is that building larger and larger models is not the only way to improve performance. This video provides a gentle intro RETRO, DeepMind’s retrieval-augmented Transformer. There are more details in the associated blog post:
Corrections:
1:15 - *knowledge-intensive tasks
---
Contents:
Introduction (0:00)
Retrieval-enhanced language models (0:40)
The Illustrated Retrieval Transformer (1:55)
World-knowledge vs. language knowledge (2:57)
The database and how to query it and retrieve information (4:45)
Feeding the prompt and retrieved information into the model (5:39)
The architecture of the model (6:00)
---
Paper:
Authors: Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, Laurent Sifre
----
Twitter:
Blog:
Mailing List:
---
More videos by Jay:
Experience Grounds Language: Improving language models beyond the world of text
Language Processing with BERT: The 3 Minute Intro (Deep learning for NLP)
Explainable AI Cheat Sheet - Five Key Categories
The Narrated Transformer Language Model
4 views
4
0
6 days ago 01:00:28 1
Vivaldi: Winter (1 hour NO ADS) - The Four Seasons| Most Famous Classical Pieces & AI Art | 432hz
6 days ago 00:00:07 1
Huge Great White Shark off Australian coast | Belived to be the biggest #greatwhite #shark #ai
6 days ago 00:03:00 1
Yaelokre - “Harpy Hare“ 𓆱 (Illustrated Song)
6 days ago 00:01:11 4
Luma Dream Machine - Fever Dreams Video Ai 2024
6 days ago 00:09:08 1
AI Interrupting Vines/Memes Compilation #2
6 days ago 00:00:56 1
Как нейросеть видит Half-Life | How AI Sees Half-Life
6 days ago 00:31:31 1
Roger Waters meets the British Palestinian Taking On Keir Starmer Ally Wes Streeting
6 days ago 00:03:24 1
BTS (방탄소년단) MAP OF THE SOUL : 7 ’Outro : Ego’ Comeback Trailer
6 days ago 00:01:15 1
Vita Sidorkina Goes Bare, Soaks Up The Sun In Nevis Shoot | Intimates | Sports Illustrated Swimsuit
6 days ago 00:05:12 1
Index AI - Bliss
6 days ago 00:04:40 1
METAL - SWEET NIGHT
6 days ago 00:01:16 1
Is This Really Me?! Xmandre Dimple Family’s DEEPBRAIN AI AVATAR Challenge!
6 days ago 00:04:00 2
Werkitten in the Minecraft | Minecraft AI audio #suno #SUNO AI
6 days ago 00:31:23 1
New Wizard | 2024 Player’s Handbook | D&D
6 days ago 00:03:50 1
Michael Scholz (Ai voice) - Oh, what a night (Ai song Udio, by Romeo Li)
6 days ago 00:13:12 1
The Last 6 Decades of AI — and What Comes Next | Ray Kurzweil | TED
6 days ago 00:04:37 1
Brick Stick House - Granny’s Good Eye (1974) [AI]
6 days ago 00:03:47 2
Red Dead Redemption 2 That’s the way it is Arthur Morgan ai cover
6 days ago 00:09:21 1
The AI Revolution: Will Robots Take Your Job?
6 days ago 01:10:38 1
Alexander Gorban, University of Leicester. Slow transients and bifurcation of attractors.
6 days ago 00:03:27 1
THE SILVERBLACK - Retaliation [LFTD Version] (Official Video) | darkTunes Music Group
6 days ago 00:04:13 1
THE SILVERBLACK - Carnivogue (Official Video) | darkTunes Music Group
6 days ago 00:03:32 1
THE SILVERBLACK - Prototype (Official Video) | darkTunes Music Group