What Do Neural Networks Really Learn? Exploring the Brain of an AI Model
Neural networks have become increasingly impressive in recent years, but there’s a big catch: we don’t really know what they are doing. We give them data and ways to get feedback, and somehow, they learn all kinds of tasks. It would be really useful, especially for safety purposes, to understand what they have learned and how they work after they’ve been trained. The ultimate goal is not only to understand in broad strokes what they’re doing but to precisely reverse engineer the algorithms encoded in their parameters. This is the ambitious goal of mechanistic interpretability. As an introduction to this field, we show how researchers have been able to partly reverse-engineer how InceptionV1, a convolutional neural network, recognizes images.
▀▀▀▀▀▀▀▀▀SOURCES & READINGS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
This topic is truly a rabbit hole. If you want to learn more about this important research and even contribute to it, check out this list of sources about mechanistic interpretability and interpretability in general we’ve compiled for you:
On Interpreting InceptionV1:
Feature visualization:
Zoom in: An Introduction to Circuits:
The Distill journal contains several articles that try to make sense of how exactly InceptionV1 does what it does:
OpenAI’s Microscope tool lets us visualize the neurons and channels of a number of vision models in great detail:
Here’s OpenAI’s Microscope tool pointed on layer Mixed3b in InceptionV1:
Activation atlases:
Transformer Circuits Thread, the spiritual successor of the circuits thread on InceptionV1. This time on transformers:
In the video, we cite “Toy Models of Superposition“:
We also cite “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning“:
More recent progress:
Mapping the Mind of a Large Language Model:
Press:
Paper in the transformers circuits thread:
Extracting Concepts from GPT-4:
Press:
Paper:
Browse features:
Language models can explain neurons in language models (cited in the video):
Press:
Paper:
View neurons:
Neel Nanda on how to get started with Mechanistic Interpretability:
Concrete Steps to Get Started in Transformer Mechanistic Interpretability:
Mechanistic Interpretability Quickstart Guide:
200 Concrete Open Problems in Mechanistic Interpretability:
More work mentioned in the video:
Progress measures for grokking via mechanistic interpretability:
Discovering Latent Knowledge in Language Models Without Supervision:
Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning:
▀▀▀▀▀▀▀▀▀PATREON, MEMBERSHIP, MERCH▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🟠 Patreon:
🔵 Channel membership:
🟢 Merch:
🟤 Ko-fi, for one-time and recurring donations:
▀▀▀▀▀▀▀▀▀SOCIAL & DISCORD▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Discord:
Reddit:
X/Twitter:
▀▀▀▀▀▀▀▀▀PATRONS & MEMBERS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
AAAA There are too many of you, and you don’t fit in the description this time! But we thank you from the bottom of our hearts. All of you, in this Google Doc:
▀▀▀▀▀▀▀CREDITS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Here are all the good doggos who worked on this video:
1 просмотр
25
8
3 месяца назад 00:03:21 1
Sleep Theory - “Fallout“
3 месяца назад 00:04:53 1
AC/DC - Thunderstruck (Official Video)
3 месяца назад 00:12:13 1
I TURN 2 HEX NUTS into a 1 Ct DIAMOND RING
3 месяца назад 00:06:34 1
Max Oazo - Feeling Me (Extended Mix)
3 месяца назад 00:00:17 1
What color eyes do you like? #likeforlikes #liked #lifestyle #lifeblogger #eyes
3 месяца назад 00:07:22 1
Most Common Exercises Done Incorrectly- Home Edition
3 месяца назад 00:04:44 1
EVERYONE who has a Washing Machine MUST watch THIS Video💥(GENIUS)🤯
3 месяца назад 00:04:12 1
In Hearts Wake - Farewell (Official Music Video)
3 месяца назад 00:03:55 1
Revenge 2
3 месяца назад 00:03:38 1
Sia & Rihanna - Beautiful People
3 месяца назад 00:01:25 1
HOW TO TELL IF YOU ARE A GOOFY OR REGULAR SNOWBOARDER
3 месяца назад 00:05:34 1
Madonna - Hung Up (Official Video) [HD]
3 месяца назад 00:20:15 1
HUGE Online Shopping Unboxing (Makeup + Skincare)
3 месяца назад 00:11:34 1
How Does He Make His Guitar Sound Like That?
3 месяца назад 00:00:36 1
Taking What’s Not Yours (Animation)
3 месяца назад 00:32:23 1
Porsche 911 Carrera S 992 | Technical Brain Damage
3 месяца назад 00:03:38 1
Paul Andi, Kina Grannis ft L.V. - Gangsta’s Paradise (cover Coolio)
3 месяца назад 00:00:55 1
Wild man make quail trap with cardboard box #wildchannel#shortvideo
3 месяца назад 00:04:43 1
Tbilisi to Kutaisi Train Ultimate Guide for Travel Tips 2023 / Georgia like European Country