Robot Control and Collaboration in Situated Instruction Following
, Cornell University
Date: Friday, February 07, 2020
Time: 12:30 PM to 1:30 PM Note: all times are in the Eastern Time Zone
Location: 32-G449 (Stata Center, Patil/Kiva Conference Room)
Event Type: Seminar
Host: Regina Barzilay, MIT CSAIL
Contact: Marcia G. Davidson, 617-253-3049, firstname.lastname@example.org
Speaker URL: None
TALK: Robot Control and Collaboration in Situated Instruction Following
I will present two projects studying the problem of learning to follow natural language instructions. I will present new datasets, a class of interpretable models for instruction following, learning methods that combine the benefits of supervised and reinforcement learning, and new evaluation protocols. In the first part, I will discuss the task of executing natural language instructions with a robotic agent. In contrast to existing work, we do not engineer formal representations of language meaning or the robot environment. Instead, we learn to directly map raw observations and language to low-level continuous control of a quadcopter drone. In the second part, I will propose the task of learning to follow sequences of instructions in a collaborative scenario, where both the user and the system execute actions in the environment and the user controls the system using natural language. To study this problem, we build CerealBar, a multi-player 3D game where a leader instructs a follower, and both act in the environment together to accomplish complex goals.
Yoav Artzi is an Assistant Professor in the Department of Computer Science and Cornell Tech at Cornell University. His research focuses on learning expressive models for natural language understanding, most recently in situated interactive scenarios. He received an NSF CAREER award, paper awards in EMNLP 2015, ACL 2017, and NAACL 2018, a Google Focused Research Award, and faculty awards from Google, Facebook, and Workday. Yoav holds a B.Sc. summa cum laude from Tel Aviv University and a Ph.D. from the University of Washington.
Created by Marcia G. Davidson at Friday, January 31, 2020 at 1:55 PM.