Thesis Defense: Hybrid Learning for Multi-Step Manipulation in Collaborative Robotics

Speaker: Claudia Perez D'Arpino , CSAIL

Date: Thursday, May 16, 2019

Time: 3:00 PM to 4:00 PM

Public: Yes

Location: Patil/Kiva (32-G449)

Event Type: Thesis Defense

Room Description:

Host: Julie Shah

Contact: Claudia Perez D'Arpino,

Relevant URL:

Speaker URL:

Speaker Photo:

Reminders to:

Reminder Subject: TALK: Thesis Defense - Claudia Perez D'Arpino: Hybrid Learning for Multi-Step Manipulation in Collaborative Robotics


I envision robots that can LEARN a model of the steps and the goal of a constrained multi-step manipulation task by observing human examples of the task, that are flexible enough to COLLABORATE with a human teammate to execute this task, and that are able to DISCOVER their own new strategy for performing the task in a manner that adapts well to unmodeled aspects of the physical world. In this thesis I formulate models and algorithms for hybrid learning, a framework in which a robot learns manipulation tasks by combining observational and self-learning, and develop a learning and collaboration workflow in the context of remote manipulation in shared autonomy. I show experimentally that this collaborative workflow significantly improves performance over other systems for remote manipulation.

I first present C-LEARN, an algorithm that enables robot learning of multi-step manipulation tasks from a single human demonstration. This work addresses the technical gap between learning from demonstrations and motion planning, effectively increasing the complexity of manipulation tasks that end users without programming experience can teach robots.

Second, I present the integration of C-LEARN into a collaborative workflow for remote manipulation. I present a user study with expert operators to evaluate four architectures for remote manipulation. I discuss the benefits of the architecture incorporating C-LEARN in terms of objective and subjective performance measures.

Finally, I present the hybrid learning framework for discovering novel strategies for multi-step manipulation, by combining learning from demonstrations and self-learning through exploration in a simulation. I demonstrate my approach by tasking a robot to manipulate blocks and assemble a stable structure. While the desired geometry is specified by the example, the underlying physics is unobservable. The robot uses Monte Carlo Tree Search (MCTS) with interleaved task and motion planning in simulation to find a robust strategy to accomplish the task.

Committee: Julie A. Shah (Chair), Leslie Kaelbling, Tomas Lozano Perez, Patrick Henry Winston, Ross A. Knepper, Chad Jenkins.

Research Areas:

Impact Areas:

This event is not part of a series.

Created by Claudia Pérez D'Arpino Email at Friday, May 10, 2019 at 9:42 AM.