DINO: Emerging Properties in Self-Supervised Vision Transformers | Paper Explained!

❤️ Become The AI Epiphany Patreon ❤️ ► In this video I cover DINO (self DIstillation with NO labels) introduced in the “Emerging Properties in Self-Supervised Vision Transformers“ paper by Facebook AI. The idea is to see whether using supervised learning was preventing transformers from showing the same kind of results in CV as they demonstrated in the NLP world (where we use self-supervised learning objectives such as (masked) language modeling). It turns out some nice properties emerge such as: * DINO-ViT learns to predict segmentation masks * features are especially of high quality for the k-NN classification ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ✅ Paper: ✅ Code: ▬