David Ha - Collective Intelligence for Deep Learning: A Survey of Recent Developments

Recorded 17 February 2022. David Ha of Google Brain presents “Collective Intelligence for Deep Learning: A Survey of Recent Developments“ at IPAM’s Mathematics of Collective Intelligence Workshop. Abstract: In the past decade, we have witnessed the rise of deep learning to dominate the field of artificial intelligence. Advances in artificial neural networks alongside corresponding advances in hardware accelerators with large memory capacity, together with the availability of large datasets enabled researchers and practitioners alike to train and deploy sophisticated neural network models that achieve state-of-the-art performance on tasks across several fields spanning computer vision, natural language processing, and reinforcement learning. However, as these neural networks become bigger, more complex, and more widely used, fundamental problems with current deep learning models become more apparent. State-of-the-art deep learning models are known to suffer from issues that range from poor rob