SVCCA: Singular Vector Canonical Correlation Analysis for Deep Understanding and Improvement

Speaker: Maithra Raghu , Google Brain, Cornell University

Date: Friday, May 26, 2017

Time: 3:00 PM to 4:00 PM Note: all times are in the Eastern Time Zone

Public: Yes

Location: 32-G449

Event Type:

Room Description:

Host: Nir Shavit and David Rolnick, MIT

Contact: Joanne Talbot Hanley, 617-253-6054, joanne@csail.mit.edu

Relevant URL:

Speaker URL: None

Speaker Photo:
None

Reminders to: theory-seminars@csail.mit.edu, seminars@csail.mit.edu

Reminder Subject: TALK: Maithra Raghu: SVCCA: Singular Vector Canonical Correlation Analysis for Deep Understanding and Improvement

Abstract: With the continuing empirical successes of deep networks, it becomes increasingly important to develop better methods for understanding training of models and the representations learned within. In this talk we propose Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less.

Research Areas:

Impact Areas:

This event is not part of a series.

Created by Joanne Talbot Hanley at Thursday, May 25, 2017 at 1:20 PM.