Thesis Defense: Neural MMO: Massively Multiagent Simulation and Learning

Speaker: Joseph Suarez , MIT CSAIL

Date: Thursday, April 11, 2024

Time: 4:30 PM to 5:30 PM Note: all times are in the Eastern Time Zone

Public: Yes

Location: 45-500A

Event Type: Thesis Defense

Room Description:

Host: Phillip Isola, MIT CSAIL

Contact: Joseph Suarez, jsuarez@mit.edu

Relevant URL:

Speaker URL: https://people.csail.mit.edu/jsuarez/

Speaker Photo:
None

Reminders to: seminars@csail.mit.edu, ei-all@lists.csail.mit.edu

Reminder Subject: TALK: Thursday 04-11-2024 Thesis Defense: "Neural MMO: Massively Multiagent Simulation and Learning"

Abstract:
Neural MMO is a massively multi-agent environment for reinforcement learning research. It is designed to push the boundaries of environment complexity while maintaining compu-
tationally efficiency for academic research. Agents in Neural MMO can forage for a variety of resources, engage in strategic combat with each other, defeat scripted enemies for loot, level-up various interdependent professions, acquire tools, weapons, equipment, etc., and exchange items on a global market. Neural MMO was among the first many-agent simulators for reinforcement learning research, and it is still unique among environments today. To my knowledge, no other project provides large agent populations, high per-agent complexity, and efficient simulation at once. These properties make Neural MMO a suitable environment for a variety of research topics in multi-agent learning that would be difficult to explore without such a simulator. The environment can process 128 agents at up to 25x real-time on a single CPU core, totaling 3,000 agent-steps per second. This speed is owed to simulation techniques borrowed from the games industry. In the course of developing Neural MMO, I made several adaptations for the specifics of reinforcement learning, such as the two-layer structure of Neural MMO’s observations and actions and the efficient internal data representation. The contributions of this work include these adapted methods as general-purpose tools for designing RL environments. Through my own experiments and from the results of a series of competitions that I hosted on Neural MMO, we have seen agents capable of long-term coherent strategies, multi-tasking across various objectives, and conditioning for specific goals. The largest discovery of this project has been the extent to which standard reinforcement learning methods with limited compute are able to solve complex tasks. Neural MMO is free and open-source software under the MIT license with comprehensive documentation at neuralmmo.github.io and a 1000+ member community Discord.

Thesis committee:
Phillip Isola, Pulkit Agrawal, Eugene Vinitsky

Zoom link: https://mit.zoom.us/my/josephsuarez

Research Areas:
AI & Machine Learning

Impact Areas:

This event is not part of a series.

Created by Phillip J Isola Email at Tuesday, April 09, 2024 at 10:38 PM.