Fall 2019 Robotics Colloquium: Debadeepta Dey (Microsoft Research)

Lecture title: Imitation-Learning with Indirect Oracles We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments. The task emulates a real-world scenario in that (a) the requester may not know how to navigate to the target objects and thus makes requests by only specifying high-level endgoals, and (b) the agent is capable of sensing when it is l