Unlocking Interaction via Pixel-Based Reverse Engineering

Speaker: Morgan Dixon , University of Washington

Date: Thursday, March 19, 2015

Time: 4:00 PM to 5:00 PM Note: all times are in the Eastern Time Zone

Refreshments: 3:45 PM

Public: Yes

Location: 32-G449

Event Type:

Room Description:

Host: Professor Randall Davis

Contact: Joanne Talbot Hanley, 617-253-4602, joanne@csail.mit.edu

Relevant URL:

Speaker URL: None

Speaker Photo:

Reminders to: seminars@csail.mit.edu

Reminder Subject: TALK: CS Special Seminar: Morgan Dixon "Unlocking Interaction via Pixel-Based Reverse Engineering" on March 19 at 4pm!

Abstract: User interface software has become a victim of its own success. Graphical event-driven systems are the dominant user interface for computational devices, realized via popular user interface toolkits. However, this success has induced a lock-in effect that demonstrably stifles innovation. For example, human-computer interaction researchers regularly propose new interaction techniques for desktop and mobile devices, enhancements for people with disabilities, better support for localization, and new collaboration facilities. However, these techniques rarely make it outside the laboratory because existing software tools cannot support them. Changing the ecosystem would require a mass adoption of new tools, and most of the applications we use today would need to be rewritten.

I will describe an approach that unlocks existing graphical application without their source code and without their cooperation, making it possible to explore and deploy many promising ideas in the context of existing real-world interfaces. Prefab is a system that reverse-engineers graphical interfaces from raw pixels. I will show how Prefab can be used to augment user interfaces with a variety of techniques, including accessibility enhancements, improved input for touch screens, UI language translation, and end-user customization. Some of these enhancements were proposed in the literature over the last two decades, but prior to Prefab they had been considered nearly impossible to deploy in practice.

Bio: Morgan Dixon is a PhD student in Computer Science & Engineering at the University of Washington working with James Fogarty. His research is centered around designing, building, and studying systems that make user interface software more flexible and extensible. He has also worked on novel interaction techniques for accessibility, dual-display mobile devices, health technologies, crossing-based interfaces, and data analysis tools. Morgan received a B.S. in Computer Science and Mathematics from the University of Maryland in 2008, and an M.S. from the University of Washington in 2010. His work on Prefab received a CHI Best Paper Award, and he was named a Microsoft Research PhD Fellow in 2011.

To see all seminars in this series, go to:

Research Areas:

Impact Areas:

See other events that are part of the CS Special Seminar Series 2015.

Created by Joanne Talbot Hanley Email at Wednesday, February 04, 2015 at 3:03 PM.