CS25 I Stanford Seminar - Self Attention and Non-parametric transformers (NPTs)

Aidan will chat about the Transformer origins and intuitions for 15 minutes. Then Neil and Jannik will take over to talk about Non-Parametric Transformers (NPTs). NPTs have just been accepted to NeurIPs and you can check them out at Aidan is a PhD student at Oxford supervised by Yarin Gal, and one of the cofounders of Cohere. He is fascinated by building massive neural networks and getting them into the hands of more engineers and researchers. Jannik is a PhD student at the University of Oxford. He is supervised by Yarin Gal and Tom Rainforth. His research interests include Bayesian Deep Learning, Active Learning, and, apparently, building non-parametric models with Transformers. Neil is a Masters by Research student at the University of Oxford, supervised by Yarin Gal. He is interested in models with relational inductive biases such as Transformers and graph neural networks, as well as Bayesian deep learning. A full list of guest lectures can be found here: