When Vision Transformers Outperform ResNets without Pretraining | Paper Explained

When Vision Transformers Outperform ResNets without Pretraining or Strong Data Augmentation paper explained. ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ✅ Paper: ✅ LinkedIn post: ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00 Key points of the paper 01:37 Key conclusions 03:00 Inductive biases and biases in a CNN 07:00 SAM explained 11:30 Possibility of heavy pruning, overfitting, sparsity, etc. 14:20 Neural tangent kernel and steepness of curvature 17:30 Results, empirical correlation between SAM and biases 19:00 Deeper look into the Hessia