Deep Learning, Transformers, and the Consequences of Scale with Oriol Vinyals - #546
Today we’re excited to kick off our annual NeurIPS, joined by Oriol Vinyals, the lead of the deep learning team at Deepmind. We cover a lot of ground in our conversation with Oriol, beginning with a look at his research agenda and why the scope has remained wide even through the maturity of the field, his thoughts on transformer models and if they will get us beyond the current state of DL, or if some other model architecture would be more advantageous. We also touch on his thoughts on the large language models craze, before jumping into his recent paper StarCraft II Unplugged: Large Scale Offline Reinforcement Learning, a follow up to their popular AlphaStar work from a few years ago. Finally, we discuss the degree to which the work that Deepmind and others are doing around games actually translates into real-world, non-game scenarios, recent work on multimodal few-shot learning, and we close with a discussion of the consequences of the level of scale that we’ve achieved thus far.
The complete show
5 views
62
15
1 month ago 00:09:12 1
Ukraine, NATO, and US: Noam Chomsky’s Analysis
1 month ago 00:18:56 1
Rethinking Climate Change. The path to a 90% emissions reduction by 2035.
1 month ago 00:13:19 1
Путешествие в самое глубокое место на планете: что скрывает Марианская впадина? | WAS
1 month ago 00:02:35 1
The Lord of the Rings: The War of the Rohirrim Trailer #1 (2024)
1 month ago 00:05:10 1
’Room of Dreams’ by John Haydock.
1 month ago 00:00:00 1
Complete Restoration of Intervertebral Discs- Healing Music for Relieving Back & Spinal Pain- 432Hz.
1 month ago 03:29:15 1
Eric Weinstein - Are We On The Brink Of A Revolution? (4K)
1 month ago 01:12:27 1
Paul McKenna on How to Power Manifest Money
1 month ago 01:55:28 1
Immediately Lose 14kg Of Visceral Fat When Listening To This Song🦴️ Healing With 528Hz Sound Therapy
1 month ago 00:39:19 9
Vivid Strings by Pulse Audio: first reaction. Logic Pro review
1 month ago 01:33:55 1
31-AUG-24 Why the Deep State Never Controlled Earth, Humanity Switches from Defense to Offense, a...
1 month ago 00:00:37 1
The Matrix (1999) From Film to Now Cast Evolution #matrix #actor #movie #evolution #top #celebrity
1 month ago 00:02:41 1
Get More Melons: Prune for Early Female Flowers🍈#melon #gardening #gardeningtips #howtogrow #plants
1 month ago 00:03:41 1
Colt Clark and the Quarantine Kids play “Yer Blues“
1 month ago 00:01:35 1
Star Wars Outlaws: Official Launch Trailer
1 month ago 00:27:15 1
Why The DEVIL Is Trying To Stress You Out! BEWARE, Chosen Ones!
1 month ago 00:20:55 1
Angels & Suicide?! “Chop Suey“ Vocal ANALYSIS and a deep dive into Serj Tankian of System Of A Down
1 month ago 00:03:13 1
How to Make Seed and Pearl Beaded Bracelet
1 month ago 02:47:04 1
294 ‒ Peak athletic performance: How to measure it and how to train for it
1 month ago 00:22:43 1
How might LLMs store facts | Chapter 7, Deep Learning
1 month ago 00:04:50 1
Lil Jon & The East Side Boyz - What U Gon’ Do (feat. Lil Scrappy) (Official Music Video)
1 month ago 00:26:10 1
Attention in transformers, visually explained | Chapter 6, Deep Learning
1 month ago 00:20:27 1
Why Meekness is the Key to Heaven: A Powerful Biblical Truth!
1 month ago 00:18:58 1
Chosen Ones, They’re Falling Apart, Your Silence Is KILLING Them!😰