DINO: Emerging Properties in Self-Supervised Vision Transformers (Facebook AI Research Explained)
#dino #facebook #selfsupervised
Self-Supervised Learning is the final frontier in Representation Learning: Getting useful features without any labels. Facebook AI’s new system, DINO, combines advances in Self-Supervised Learning for Computer Vision with the new Vision Transformer (ViT) architecture and achieves impressive results without any labels. Attention maps can be directly interpreted as segmentation maps, and the obtained representations can be used for image retrieval and zero-shot k-nearest neighbor classifiers (KNNs).
OUTLINE:
0:00 - Intro & Overview
6:20 - Vision Transformers
9:20 - Self-Supervised Learning for Images
13:30 - Self-Distillation
15:20 - Building the teacher from the student by moving average
16:45 - DINO Pseudocode
23:10 - Why Cross-Entropy Loss?
28:20 - Experimental Results
33:40 - My Hypothesis why this works
38:45 - Conclusion & Comments
Paper:
Blog:
2 views
30
6
5 months ago 00:15:00 1
PLA Tank Combiner Sembo 203160-203163 ⚡️ How to make Robot LEGO Transformers Combiner build tutorial
6 months ago 00:15:01 1
Guided Breathing Exercise To Help Feel Positivity I 3 Rounds I One Love
9 months ago 00:23:46 1
There Go the Brides | Samurai | Full Episode | S18 | E08 | Power Rangers Official
9 months ago 00:21:32 1
Invasion of the Body Switcher | In Space | Full Episode | S06 | E19 | Power Rangers Official
9 months ago 00:00:59 6
”Jurassic Rampage” Cinematic Trailer
10 months ago 02:00:56 1
Silicon Dream - Best Hits & Project 1987-2000
10 months ago 00:21:25 1
Forever Red | Power Rangers Wild Force | Full Episode | E34 | Power Rangers Official
1 year ago 00:10:22 1
What’s behind the wave of coups in Africa? | Start Here
1 year ago 00:14:24 1
LEGO Robot Takeda Shingen ( Sluban M38 - B1183 ) ⚡️ Speed build How to make a Robot Samurai Kai
1 year ago 00:09:25 27
More Globe Lies Exposed by FLIGHT ROUTES
1 year ago 00:07:25 1
Team Up with the Retro Rangers! | Operation Overdrive | Power Rangers Official
1 year ago 16:53:46 3
Spyro Trilogy - Complete 337% Walkthrough - All Collectibles (Longplay)
1 year ago 00:05:23 1
Starving, sick, homeless dog transforms into something amazing just in time for Eldad’s birthday! 😊
1 year ago 00:11:14 2
Mecha Domination: Rampage - Early Access USA Gameplay Android APK iOS
1 year ago 00:00:42 42
Mighty Morphin Power Rangers: Once & Always | “Once A Ranger“ Sneak Peek | Power Rangers Official
1 year ago 00:42:34 1
TRANSFORMERS: AGE OF EXTINCTION | MOVIE REACTION | Our First Time Watching | UNDERRATED STORY 🤯😨
1 year ago 00:33:52 1
Top 10 Final Battles | Power 10s | Power Rangers Official
2 years ago 00:02:24 1
EXOPRIMAL Exosuit Trailer (2023) 4K UHD
2 years ago 00:02:44 2
EXOPRIMAL Trailer (2023) Exosuit Based Combat Games | 4K UHD
2 years ago 00:00:15 2
Ramon Dino🦖 Legendary Arnold Classic Posing🥇#bodybuilding #shorts #ramondino #arnoldclassic #maromba
2 years ago 01:04:01 1
THE BEST OF ITALO DISCO - Produced by Klaus Munzert
2 years ago 00:02:22 34
Mighty Morphin Power Rangers: Once & Always 30th Anniversary Special – Rangers Reunited
2 years ago 01:12:01 4
WAX2WAX 49 (Assaf Naftali) (Deep Techno)
2 years ago 00:47:46 1
Dungeon Synth, A Long Introduction | Esoteric Internet