Robot Control and Collaboration in Situated Instruction Following

Speaker: Yoav Artzi , Cornell University

Date: Friday, February 07, 2020

Time: 12:30 PM to 1:30 PM Note: all times are in the Eastern Time Zone

Public: Yes

Location: 32-G449 (Stata Center, Patil/Kiva Conference Room)

Event Type: Seminar

Room Description:

Host: Regina Barzilay, MIT CSAIL

Contact: Marcia G. Davidson, 617-253-3049, marcia@csail.mit.edu

Relevant URL:

Speaker URL: None

Speaker Photo:
None

Reminders to: seminars@csail.mit.edu, rbg@csail.mit.edu

Reminder Subject: TALK: Robot Control and Collaboration in Situated Instruction Following

ABSTRACT:
I will present two projects studying the problem of learning to follow natural language instructions. I will present new datasets, a class of interpretable models for instruction following, learning methods that combine the benefits of supervised and reinforcement learning, and new evaluation protocols. In the first part, I will discuss the task of executing natural language instructions with a robotic agent. In contrast to existing work, we do not engineer formal representations of language meaning or the robot environment. Instead, we learn to directly map raw observations and language to low-level continuous control of a quadcopter drone. In the second part, I will propose the task of learning to follow sequences of instructions in a collaborative scenario, where both the user and the system execute actions in the environment and the user controls the system using natural language. To study this problem, we build CerealBar, a multi-player 3D game where a leader instructs a follower, and both act in the environment together to accomplish complex goals.

BIO:
Yoav Artzi is an Assistant Professor in the Department of Computer Science and Cornell Tech at Cornell University. His research focuses on learning expressive models for natural language understanding, most recently in situated interactive scenarios. He received an NSF CAREER award, paper awards in EMNLP 2015, ACL 2017, and NAACL 2018, a Google Focused Research Award, and faculty awards from Google, Facebook, and Workday. Yoav holds a B.Sc. summa cum laude from Tel Aviv University and a Ph.D. from the University of Washington.

Research Areas:

Impact Areas:

This event is not part of a series.

Created by Marcia G. Davidson Email at Friday, January 31, 2020 at 1:55 PM.