Hot Topics in Computing: Matei Zaharia, Going Beyond Models to Reliable AI Systems using LLMs

Speaker: Matei Zaharia

Date: Wednesday, October 11, 2023

Time: 5:00 PM to 6:00 PM Note: all times are in the Eastern Time Zone

Public: Yes

Location: 32-123

Event Type: Seminar

Room Description: Kirsch Auditorium

Host: Daniela Rus, MIT SCC & CSAIL

Contact: Lauralyn M. Smith, events@csail.mit.edu

Relevant URL:

Speaker URL: None

Speaker Photo:
None

Reminders to: seminars@csail.mit.edu

Reminder Subject: TALK: Hot Topics in Computing: Matei Zaharia

Abstract:
While large language models have unlocked amazing results for AI, it is difficult to use them to build reliable end-to-end applications. LLMs alone offer limited interpretability and control; their knowledge is locked in at training time; and they are highly expensive to train and execute. An emerging alternative is to compose LLM invocations with external tools such as retrieval (search) and APIs into an AI system for performing a particular task. How can we express, train and systematically improve these multi-component AI systems? Naive approaches such as agent loops often perform very poorly, but I will present research that tackles this problem systematically to obtain state-of-the-art results in several domains with minimal developer effort. In particular, our DSPy open source framework can take a pipeline of LM and tool calls written in a PyTorch-like API and automatically discover prompts, few-shot examples or even fine tuned models to execute it effectively, using an inexpensive bootstrapping process. It automatically builds strong pipelines for reasoning (GSM8K), multi hop open domain question answering (HotpotQA), and other tasks while offering strong controls for developers to improve performance, matching hand built systems with tens of thousands of characters of manually engineered prompts. I’ll also talk about how I see these multi-component AI systems being adopted in industry and open questions in this field.

Bio:
Matei Zaharia is a Cofounder and CTO at Databricks as well as an Associate Professor of Computer Science at UC Berkeley. He started the Apache Spark cluster computing project during his PhD at UC Berkeley in 2009, which is now one of the most widely used systems for parallel data processing and machine learning, and he has worked broadly on other widely used data and AI software, including MLflow, Delta Lake, Dolly, and ColBERT. His most recent research is about combining large language models (LLMs) with external tools to build powerful and reliable end-to-end systems. Matei’s research was recognized through the 2014 ACM Doctoral Dissertation Award, an NSF CAREER Award, and the US Presidential Early Career Award for Scientists and Engineers (PECASE).

Research Areas:

Impact Areas:

This event is not part of a series.

Created by Lauralyn M. Smith at Monday, September 25, 2023 at 4:38 PM.