Title: Scaling Personalized Feedback for Programming Assignments using Program Synthesis

Speaker: Gustavo Soares, Federal University of Campina Grande, Brazil , Boern Hartmann, UC Berkeley

Date: Friday, April 21, 2017

Time: 11:00 AM to 12:00 PM Note: all times are in the Eastern Time Zone

Refreshments: 10:45 AM

Public: Yes

Location: 32-G449 Patil Kiva

Event Type:

Room Description:

Host: Martin Rinard, CSAIL

Contact: Mary McDavitt, 617-253-9620, mmcdavit@csail.mit.edu

Relevant URL: www.dsc.ufcg.edu.br/~gsoares/

Speaker URL: None

Speaker Photo:
None

Reminders to: seminars@csail.mit.edu, pac@lists.csail.mit.edu

Reminder Subject: TALK: Scaling Personalized Feedback for Programming Assignments using Program Synthesis

Abstract:

In large introductory programming classes, teacher feedback on individual incorrect student submissions is often infeasible. In this talk, we will present Refazer, a technique for automatically learning program transformations, and two systems based on Refazer that allow teachers to give personalized, verified feedback at scale: MistakeBrowser and FixPropagator.

Refazer builds on the observation that code edits performed by students can be used as input-output examples for learning program transformations. Example edits may share the same structure but involve different variables and subexpressions, which must be generalized in a transformation at the right level of abstraction. To learn transformations, Refazer leverages state-of-the-art programming-by-example methodology using the following key components: (a) a novel domain-specific language (DSL) for describing program transformations, (b) domain-specific deductive algorithms for efficiently synthesizing transformations in the DSL, and (c) functions for ranking the synthesized transformations. In our evaluation conducted on 4 programming tasks performed by 720 students, our technique helped to fix incorrect submissions for 87% of the students.

Refazer is capable of fixing student bugs and generating hints automatically, but it lacks the deep domain knowledge of a teacher and it can generate functionally correct but stylistically poor fixes. MistakeBrowser and FixPropagator use Refazer to learn bug-fixing code transformations and then cluster incorrect submissions by the transformations that correct them. Teachers can write feedback about a single submission or a cluster of submissions and propagate the feedback to all other submissions that can be fixed by the same transformation. Two studies suggest this approach helps teachers better understand student bugs and write reusable feedback that scales to a massive introductory programming classroom.

BIO:

Gustavo Soares is an Assistant Professor in Department of Computing and Systems at the Federal University of Campina Grande, Brazil. Previously, he worked as a postdoc at UC Berkeley with Bjoern Hartmann on program synthesis techniques for helping students and developers, and worked with Sumit Gulwani at Microsoft Research on program synthesis and interaction models for automating data wrangling tasks. His research interests include programming language, HCI, software engineering, and education. His research has received important awards such as the ACM SIGPLAN John Vlissides Award, the ACM Student Research Competition Gold Medal at SPLASH, and the CAPES Award for Best CS PhD Thesis in Brazil. He completed his PhD in Computer Science at Federal under supervision of Rohit Gheyi at Federal University of Campina Grande in 2014.

For more information visit www.dsc.ufcg.edu.br/~gsoares/

BIO:

Bjoern Hartmann is an Associate Professor in EECS at UC Berkeley, where he is the faculty director of the new Jacobs Institute for Design Innovation. His research in Human-Computer Interaction focuses on novel design, prototyping, and implementation tools for the era of post-personal computing. His group investigates how better software and hardware can facilitate the exploration of interactive devices that leverage novel form factors and technologies. He also investigates how software can help students, designers, and makers to learn and share their expertise online. His research has received numerous Best Paper Awards at top Human-Computer Interaction conferences, a Sloan Fellowship, an Okawa Research Award and an NSF CAREER Award. He received both the Diane S. McEntyre Award and the Jim and Donna Gray Faculty Award for Excellence in Teaching. He completed his PhD in Computer Science at Stanford University in 2009, and received degrees in Digital Media Design, Communication, and Computer and Information Science from the University of Pennsylvania in 2002.

For more information visit http://www.cs.berkeley.edu/~bjoern

Research Areas:

Impact Areas:

This event is not part of a series.

Created by Mary McDavitt at Wednesday, April 19, 2017 at 9:15 AM.