Tensor Programs: A Swiss-Army Knife for Nonlinear Random Matrix Theory of Deep Learning and Beyond

Speaker: Greg Yang , Microsoft Research

Date: Wednesday, May 29, 2019

Time: 4:00 PM to 5:00 PM

Public: Yes

Location: 32-G575

Event Type:

Room Description:

Host: Govind Ramnarayan, Quanquan Liu, Sitan Chen, Nikhil Vyas, MIT CSAIL

Contact: Rebecca Yadegar, ryadegar@csail.mit.edu

Relevant URL:

Speaker URL: None

Speaker Photo:
None

Reminders to: seminars@csail.mit.edu, theory-semianrs@csail.mit.edu

Reminder Subject: TALK: Greg Yang:Tensor Programs: A Swiss-Army Knife for Nonlinear Random Matrix Theory of Deep Learning and Beyond

Abstract: The resurgence of neural networks has revolutionized artificial intelligence since 2010. Luckily for mathematicians and statistical physicists, the study of large random network scaling limits, which can be thought of as *nonlinear* random matrix theory, is both practically important and mathematically interesting. We describe several problems in this setting and develop a new comprehensive framework, called “tensor
programs,” for solving these problems. This framework can be thought of as an automatic tool to derive the behavior of computation graphs with large matrices, as used in neural network computation. It is very general, and from it we also obtain new proofs of the semicircle and the Marchenko-Pastur laws. Thus, “tensor programs” is broadly useful to linear
and nonlinear random matrix theory alike, and we hope it will be adopted as a standard tool.
This talk presents the work arXiv:1902.04760.

Research Areas:
Algorithms & Theory

Impact Areas:

See other events that are part of the Algorithms & Complexity Seminars 2018-2019.

Created by Rebecca Yadegar Email at Monday, May 20, 2019 at 12:28 PM.