How large language models work, a visual intro to transformers | Chapter 5, Deep Learning
Breaking down how Large Language Models work
Instead of sponsored ad reads, these lessons are funded directly by viewers:
---
Here are a few other relevant resources
Build a GPT from scratch, by Andrej Karpathy
If you want a conceptual understanding of language models from the ground up, @vcubingx just started a short series of videos on the topic:
If you’re interested in the herculean task of interpreting what these large networks might actually be doing, the Transformer Circuits posts by Anthropic are great. In particular, it was only after reading one of these that I started thinking of the combination of the value and output matrices as being a combined low-rank map from the embedding space to itself, which, at least in my mind, made things much clearer than other sources.
Site with exercises related to ML programming and GPTs
History of language models by Brit Cruise, @ArtOfTheProblem
An early paper on how directions in embedding spaces have meaning:
---
Timestamps
0:00 - Predict, sample, repeat
3:03 - Inside a transformer
6:36 - Chapter layout
7:20 - The premise of Deep Learning
12:27 - Word embeddings
18:25 - Embeddings beyond words
20:22 - Unembedding
22:22 - Softmax with temperature
26:03 - Up next
1 view
65
15
6 months ago 00:24:46 1
Why π is in the normal distribution (beyond integral tricks)
6 months ago 00:02:06 20
People are running away in panic from huge rocks from the sky! Huge hail in Uruguay
6 months ago 00:19:51 1
🇫🇷‘I feel safer in RUSSIA than in France’ | FRENCH lady Gabrielle Duvoisin @GabrielleDuvoisin
6 months ago 00:20:41 1
Tulip Flower Canes | Polymer Clay Cane Tutorial
6 months ago 00:13:59 1
How to Macrame a Half Mandala Wall Hanging
6 months ago 00:10:01 1
№ 1 для снижения сахара в крови! Снизить сахар в крови без таблеток и восстановить здоровье!
6 months ago 00:15:35 1
How Russia produces 3 million artillery rounds per year
6 months ago 00:36:40 1
Building the ULTIMATE Vintage City Street DIORAMA - REALISTIC Miniature Model Scenery
6 months ago 00:04:54 1
Making and tuning a perfect sounding 2 octaves Wooden Tongue Drum (by a musician & for musicians).
6 months ago 00:39:58 1
Recycle Newspaper - Large Basket | Waste Material Craft | Paper Weaving | Wicker Craft | DIY Storage
6 months ago 00:06:44 1
Clivus Multrum 8 Next Gen | Continuous Composting Toilet System
6 months ago 00:36:43 1
EK is Imploding: Not Paying Employees, Partners, & Suppliers | Investigative Report
6 months ago 00:43:03 1
“They will CONTROL everything you do very soon.“ Journalist Whitney Webb | Redacted News
6 months ago 00:03:07 4
Most Brown Bear Population by Country 2022
6 months ago 00:09:06 1
DIY Easy Shoulder Bag | How to make a Large Messanger Bag Tutorial [sewingtimes]
6 months ago 00:10:15 1
Don’t Waste Time For Me! Poor Dog Tearfully Gave Up and Waited for The End
6 months ago 00:03:45 1
Talking Heads - Once in a Lifetime (Official Video)
6 months ago 00:09:48 1
how to make denim zipper shoulder bag from old jeans waist bands,shoulder bag tutorial,sling bag diy
6 months ago 02:01:36 1
(Áudio) O que é Maoísmo? Um panorama histórico e teórico (part. João Pedro Fragoso)
6 months ago 00:12:00 1
Drum Machine Trick: How to import loops from a 100,000+ MIDI clip library into almost any sequencer
6 months ago 00:15:56 1
Photography props for CHEAP. Where to buy and how to DYI props for under $5 each
6 months ago 11:55:00 1
Open All the Doors of Prosperity, Remove All Barriers, Receive Large Amounts of Money Non-stop 777Hz
6 months ago 00:42:10 1
Infinite Energy Generator 10Kw Triphasic - Liberty Engine 1.1. LONG VERSION
6 months ago 00:17:33 1
Triphasic Infinite Energy Generator 10Kw 230V - Liberty Engine 1.1