Stanford Seminar - Information Theory of Deep Learning

EE380: Computer Systems Colloquium Seminar Information Theory of Deep Learning Speaker: Naftali Tishby, Computer Science, Hebrew Univerisity I will present a novel comprehensive theory of large scale learning with Deep Neural Networks, based on the correspondence between Deep Learning and the Information Bottleneck framework. The new theory has the following components: 1. rethinking Learning theory; I will prove a new generalization bound, the input-compression bound, which shows that compression of the