Andrej Karpathy Building makemore Part 4: Becoming a Backprop Ninja
🎯 Загружено автоматически через бота:
🚫 Оригинал видео:
📺 Данное видео принадлежит каналу «Andrej Karpathy» (@AndrejKarpathy). Оно представлено в нашем сообществе исключительно в информационных, научных, образовательных или культурных целях. Наше сообщество не утверждает никаких прав на данное видео. Пожалуйста, поддержите автора, посетив его оригинальный канал.
✉️ Если у вас есть претензии к авторским правам на данное видео, пожалуйста, свяжитесь с нами по почте support@, и мы немедленно удалим его.
📃 Оригинальное описание:
We take the 2-layer MLP (with BatchNorm) from the previous video and backpropagate through it manually without using PyTorch autograd’s (): through the cross entropy loss, 2nd linear layer, tanh, batchnorm, 1st linear layer, and the embedding table. Along the way, we get a strong intuitive understanding about how gradients flow backwards through the compute graph and on the level of efficient Tensors, not just individual scalars like in micrograd. This helps build competence and intuition around how neural nets are optimized and sets you up to more confidently innovate on and debug modern neural networks.
!!!!!!!!!!!!
I recommend you work through the exercise yourself but work with it in tandem and whenever you are stuck unpause the video and see me give away the answer. This video is not super intended to be simply watched. The exercise is here:
!!!!!!!!!!!!
Links:
makemore on github:
jupyter notebook I built in this video:
collab notebook:
my website:
my twitter:
our Discord channel:
Supplementary links:
Yes you should understand backprop:
BatchNorm paper:
Bessel’s Correction:
Bengio et al. 2003 MLP LM
Chapters:
intro: why you should care & fun history
starter code
exercise 1: backproping the atomic compute graph
brief digression: bessel’s correction in batchnorm
exercise 2: cross entropy loss backward pass
exercise 3: batch norm layer backward pass
exercise 4: putting it all together
outro
2 views
0
0
4 years ago 00:11:11 67
PyTorch at Tesla - Andrej Karpathy, Tesla
4 years ago 00:30:09 338
Andrej Karpathy - AI for Full-Self Driving
11 months ago 00:59:48 16
Введение в большие языковые модели от Andrej Karpathy
3 years ago 00:24:15 58
Andrej Karpathy Talking about AI and Neural Networks at Tesla for Full Self Driving
4 years ago 00:27:39 61
Tesla AI Andrej Karpathy on Scalability in Autonomous Driving
2 years ago 03:28:48 22
Andrej Karpathy: Tesla AI, Self-Driving, Optimus, Aliens, and AGI | Lex Fridman Podcast #333
4 years ago 00:28:51 18
Scalability in Autonomus Driving Workshop | CVPR’20 Keynote | Andrej Karpathy | Tesla
9 years ago 00:15:47 62
Andrej Karpathy, Research Scientist, OpenAI - RE•WORK Deep Learning Summit 2016 #reworkDL
4 years ago 01:30:56 1
Recent Breakthroughs in AI with Andrej Karpathy and Lex Fridman
5 years ago 00:09:16 10
Tesla Neural Network Multi-Task Learning Summarized | Andrej Karpathy
3 years ago 00:37:47 1
Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles
8 years ago 01:25:17 44
Deep Learning for Computer Vision (Andrej Karpathy, OpenAI)
4 years ago 00:10:32 29
AI for Full-Self Driving by Andrej Karpathy in 10 Minutes
3 years ago 00:15:11 2
’s Heroes of Deep Learning: Andrej Karpathy
7 years ago 00:15:11 72
Heroes of Deep Learning: Andrew Ng interviews Andrej Karpathy
3 years ago 00:23:47 10
Self-driving from VISION ONLY - Tesla’s self-driving progress by Andrej Karpathy (Talk Analysis)
4 years ago 00:17:55 1
Building the Software 2 0 Stack (Andrej Karpathy)
1 year ago 00:05:48 3
Advice for machine learning beginners | Andrej Karpathy and Lex Fridman
9 years ago 00:03:44 145
NeuralTalk and Walk, recognition, text description of the image while walking
4 years ago 00:13:01 3
Explaining Andrej (Tesla’s AI lead)- Part 1: Deep Learning
10 years ago 00:53:27 18
Automated Image Captioning with ConvNets and Recurrent Nets
7 months ago 00:26:53 1
Vedal & Neuro Build A Language Model From Scratch
11 months ago 00:14:07 1
“Что в имени тебе моем?“ Учимся генерировать новые имена у звездного разработчика Tesla и OpenAI.