How might LLMs store facts | Chapter 7, Deep Learning
Unpacking the multilayer perceptrons in a transformer, and how they may store facts
Instead of sponsored ad reads, these lessons are funded directly by viewers:
An equally valuable form of support is to share the videos.
AI Alignment forum post from the Deepmind researchers referenced at the video’s start:
Anthropic posts about superposition referenced near the end:
Sections:
0:00 - Where facts in LLMs live
2:15 - Quick refresher on transformers
4:39 - Assumptions for our toy example
6:07 - Inside a multilayer perceptron
15:38 - Counting parameters
17:04 - Superposition
21:37 - Up next
------------------
These animations are largely made using a custom Python library, manim. See the FAQ comments here:
#manim
All code for specific videos is visible here:
The music is by Vincent Rubinetti.
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. If you’re reading the bottom of a video description, I’m guessing you’re more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly.
Mailing list:
Twitter:
Instagram:
Reddit:
Facebook:
Patreon:
Website:
1 view
435
117
5 months ago 00:03:29 1
Spain vs. Colombia vs. Mexico vs. Argentina | Spanish Word Differences
5 months ago 00:19:48 1
Listening to Yakuza 0 Soundtrack Be Like...
5 months ago 00:11:22 1
POWERLESS: Biden can’t save Netanyahu from arrest. Here’s why.
5 months ago 00:10:19 1
Iran JUST SHOWED Insane Naval Power That Plan to Defeat Israel and US Navy
5 months ago 00:03:35 1
Linkin Park - | GUITAR & BASS ONLY + TABS on screen | HOW TO PLAY
5 months ago 00:26:10 1
Attention in transformers, visually explained | Chapter 6, Deep Learning
5 months ago 00:11:19 1
Human Parasite Tierlist
5 months ago 00:05:29 1
Madonna - Ghosttown
5 months ago 00:03:16 1
Snoop Dogg & DJ Drama ft. Stressmatic - I’m From 21st Street (Official Music Video)
5 months ago 00:06:27 1
Justice League: Crisis On Infinite Earths Part Two | Extended Preview | Warner Bros. Entertainment
5 months ago 00:00:59 1
when it rains it pours
5 months ago 00:03:54 1
Bring Me The Horizon - Sleepwalking | GUITAR & DRUM TRACK | mike KidLazy
5 months ago 01:04:59 1
Enchanted Abandoned Princess’ Castle in France - Now Turning Into A Looter’s Paradise!
5 months ago 00:08:13 1
DATA LEAK: How is Israel accessing WhatsApp to find targets in Gaza?
5 months ago 00:04:11 1
The Raconteurs – Old Enough (Official Music Video)
5 months ago 00:08:44 1
Swedish Vallhund - Top 10 Facts
5 months ago 00:35:50 1
How I Use the Antelope Audio CONTROL PANEL
5 months ago 00:05:06 5
Magnitude 4.1 Earthquake Hits Los Angeles California - May 1st, 2024
5 months ago 01:02:50 1
Spring Blooms • Beautiful Instrumental Piano Music • Floral Inspiration Ideas • Quintessential Home
5 months ago 00:04:59 1
Billie Eilish - Therefore I Am (Official Music Video)
5 months ago 00:01:36 1
Top 20 Beautiful Actress From Different Countries
5 months ago 01:05:52 1
HOW TO LEARN ENGLISH
5 months ago 00:04:51 1
JINJER - Home Back (Official Video 4K) | Napalm Records
5 months ago 00:02:36 1
Hell’s Greatest Mom (Hell’s Greatest Dad Female Cover) | Hazbin Hotel | Alastor X Lucifer