How large language models work, a visual intro to transformers | Chapter 5, Deep Learning
Breaking down how Large Language Models work
Instead of sponsored ad reads, these lessons are funded directly by viewers:
---
Here are a few other relevant resources
Build a GPT from scratch, by Andrej Karpathy
If you want a conceptual understanding of language models from the ground up, @vcubingx just started a short series of videos on the topic:
If you’re interested in the herculean task of interpreting what these large networks might actually be doing, the Transformer Circuits posts by Anthropic are great. In particular, it was only after reading one of these that I started thinking of the combination of the value and output matrices as being a combined low-rank map from the embedding space to itself, which, at least in my mind, made things much clearer than other sources.
Site with exercises related to ML programming and GPTs
History of language models by Brit Cruise, @ArtOfTheProblem
An early paper on how directions in embedding spaces have meaning:
---
Timestamps
0:00 - Predict, sample, repeat
3:03 - Inside a transformer
6:36 - Chapter layout
7:20 - The premise of Deep Learning
12:27 - Word embeddings
18:25 - Embeddings beyond words
20:22 - Unembedding
22:22 - Softmax with temperature
26:03 - Up next
1 view
65
15
2 months ago 00:16:48 1
БАЛЫК из ЩУКИ за СУТКИ ! ЩУКА ВЯЛЕНАЯ | КАК ВЯЛИТЬ ЩУКУ ЛУЧШИЙ РЕЦЕПТ | ЛУЧШАЯ ЗАКУСКА ИЗ РЫБЫ
2 months ago 00:11:09 1
ТЮЛЬПАНЫ ИЗ ЗЕФИРА 😍💐 Одной НАСАДКОЙ | Наталья Деревянко
2 months ago 00:09:04 9
Sample Preparation for MALDI-TOF Mass Spectrometry
2 months ago 00:23:55 1
One Day in Life of the village Family in Russia. Autumn In The Russian North
2 months ago 00:10:58 1
“ALCOHOL IS DYNAMITE“ 1960s DRUNK DRIVING DRIVER’S ED SCARE FILM 13144
2 months ago 00:19:52 1
Crochet Puff Flowers & How to Join As You Go - NO SEWING! 🌸
2 months ago 00:02:17 1
* КОГДА-НИБУДЬ ЗАКОНЧИТСЯ ВОЙНА * стихи поэта Таня Витязь Белгород
2 months ago 00:05:07 2
How to make an Aztec Death Whistle (without clay)
2 months ago 00:03:05 1
The Ultimate Guide on How to Find Winning Products❗️❗️
2 months ago 00:11:11 1
How American Farmer Ranchers Raise 5.2 Million Sheep - FAFY Farm