- The topology of representat...
- Edit Event
- Cancel Event
- Preview Reminder
- Send Reminder
- Other events happening in April 2019

## The topology of representation teleportation, regularized Oja's rule, and weight symmetry

**Speaker:**
Dr. Jon Bloom
, Broad Institute

**Date:**
Tuesday, April 02, 2019

**Time:**
4:00 PM to 5:00 PM

**Public:**
Yes

**Location:**
46-3002

**Event Type:**
Seminar

**Room Description:**
Singleton Auditorium (46-3002), 43 Vassar St, Cambridge, MA 02139

**Host:**
Boris Katz & Tomaso Poggio

**Contact:**
Kathleen Sullivan, kdsulliv@csail.mit.edu

**Relevant URL:**
https://cbmm.mit.edu/news-events/events/brains-minds-machines-seminar-series-topology-representation-teleportation

**Speaker URL:**
https://www.broadinstitute.org/bios/jonathan-bloom

**Speaker Photo:**

**Reminders to:**
seminars@csail.mit.edu

**Reminder Subject:**
TALK: The topology of representation teleportation, regularized Oja's rule, and weight symmetry

Abstract: When trained to minimize reconstruction error, a linear autoencoder (LAE) learns the subspace spanned by the top principal directions but cannot learn the principal directions themselves. In this talk, I'll explain how this observation became the focus of a project on representation learning of neurons using single-cell RNA data. I'll then share how this focus led us to a satisfying conversation between numerical analysis, algebraic topology, random matrix theory, deep learning, and computational neuroscience. We'll see that an L2-regularized LAE learns the principal directions as the left singular vectors of the decoder, providing a simple and scalable PCA algorithm related to Oja's rule. We'll use the lens of Morse theory to smoothly parameterize all LAE critical manifolds and the gradient trajectories between them; and see how algebra and probability theory provide principled foundations for ensemble learning in deep networks, while suggesting new algorithms. Finally, we'll come full circle to neuroscience via the "weight transport problem" (Grossberg 1987), proving that L2-regularized LAEs are symmetric at all critical points. This theorem provides local learning rules by which maximizing information flow and minimizing energy expenditure give rise to less-biologically-implausible analogues of backproprogation, which we are excited to explore in vivo and in silico. Joint learning with Daniel Kunin, Aleksandrina Goeva, and Cotton Seed.

Project resources: https://github.com/danielkunin/Regularized-Linear-Autoencoders

Short Bio: Jon Bloom is an Institute Scientist at the Stanley Center for Psychiatric Research within the Broad Institute of MIT and Harvard. In 2015, he co-founded the Models, Inference, and Algorithms Initiative and a team (Hail) building distributed systems used throughout academia and industry to uncover the biology of disease. In his youth, Jon did useless math at Harvard and Columbia and learned useful math by rebuilding MITâ€™s Intro to Probability and Statistics as a Moore Instructor and NSF postdoc. These days, he is exuberantly surprised to find the useless math may be useful after all.

**Research Areas:**

Algorithms & Theory, AI & Machine Learning

**Impact Areas:**

Created by Kathleen Sullivan at Monday, April 01, 2019 at 4:01 PM.