This is a collection of Multi-Agent Reinforcement Learning (MARL) papers. Each category is a potential start point for you to start your research. Some papers are listed more than once because they belong to multiple categories.
For MARL papers with code and MARL resources, please refer to MARL Papers with Code and MARL Resources Collection.
I will continually update this repository and I welcome suggestions. (missing important papers, missing categories, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months.
Update (2026): A large batch of recent papers (2022–2026) has been added across most categories, plus three new sections — LLM-based Multi-Agent, Zero-Shot Coordination and Generalization — and a Recent Surveys (2022–2026) list.
This repository is not for commercial purposes.
My email: chenhao915@mails.ucas.ac.cn
- Reviews
- Environments
- Dealing With Credit Assignment Issue
- Policy Gradient
- Communication
- Emergent
- Opponent Modeling
- Game Theoretic
- Hierarchical
- Ad Hoc Teamwork
- League Training
- Curriculum Learning
- Mean Field
- Transfer Learning
- Meta Learning
- Fairness
- Exploration
- Graph Neural Network
- Model-based
- NAS
- Safe Multi-Agent Reinforcement Learning
- From Single-Agent to Multi-Agent
- Discrete-Continuous Hybrid Action Spaces / Parameterized Action Space
- Role
- Diversity
- Sparse Reward
- Large Scale
- DTDE
- Decision Transformer
- Offline MARL
- Generalization
- Adversarial
- LLM-based Multi-Agent
- Zero-Shot Coordination
- Generalization
- Multi-Agent Path Finding
- To be Categorized
- TODO
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
- Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
- A Review of Cooperative Multi-Agent Deep Reinforcement Learning
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
- A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
- Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
- A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems
- Multi-agent Reinforcement Learning: A Comprehensive Survey
- A Survey of Progress on Cooperative Multi-Agent Reinforcement Learning in Open Environments
- A Survey of Multi-Agent Deep Reinforcement Learning with Communication (AAMAS 2024)
- A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives
- A Survey on Large Language Model based Multi-Agent Systems: Recent Advances and New Frontiers in Application
- Multi-Agent Collaboration Mechanisms: A Survey of LLMs
- A Survey of Safe Reinforcement Learning and Constrained MDPs: Single-Agent and Multi-Agent Safety
- Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
- If multi-agent learning is the answer, what is the question?
- Multiagent learning is not the answer. It is the question
- Is multiagent deep reinforcement learning the answer or the question? A brief survey Note that A Survey and Critique of Multiagent Deep Reinforcement Learning is an updated version of this paper with the same authors.
- Evolutionary Dynamics of Multi-Agent Learning: A Survey
- (Worth reading although they're not recent reviews.)
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| COMA：Counterfactual Multi-Agent Policy Gradients | https://github.com/oxwhirl/pymarl | AAAI | 2018 |
| LiCA：Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning | https://github.com/mzho7212/LICA | NIPS | 2020 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Bayesian Opponent Exploitation in Imperfect-Information Games | IEEE Conference on Computational Intelligence and Games | 2018 | |
| LOLA：Learning with Opponent-Learning Awareness | AAMAS | 2018 | |
| Variational Autoencoders for Opponent Modeling in Multi-Agent Systems | 2020 | ||
| Stable Opponent Shaping in Differentiable Games | 2018 | ||
| Opponent Modeling in Deep Reinforcement Learning | https://github.com/hhexiy/opponent | ICML | 2016 |
| Game Theory-Based Opponent Modeling in Large Imperfect-Information Games | AAMAS | 2011 | |
| Agent Modelling under Partial Observability for Deep Reinforcement Learning | NIPS | 2021 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| AlphaStar：Grandmaster level in StarCraft II using multi-agent reinforcement learning | Nature | 2019 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Mean Field Multi-Agent Reinforcement Learning | ICML | 2018 | |
| Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning | The world wide web conference | 2019 | |
| Bayesian Multi-type Mean Field Multi-agent Imitation Learning | NIPS | 2020 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems | Journal of Artificial Intelligence Research | 2019 | |
| Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning | 2020 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning | ICML | 2021 | |
| Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments | 2017 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| FEN：Learning Fairness in Multi-Agent Systems | NIPS | 2019 | |
| Fairness in Multiagent Resource Allocation with Dynamic and Partial Observations | AAMAS | 2018 | |
| Fairness in Multi-agent Reinforcement Learning for Stock Trading | 2019 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| EITI/EDTI：Influence-Based Multi-Agent Exploration | https://github.com/TonghanWang/EITI-EDTI | ICLR | 2020 |
| Cooperative Exploration for Multi-Agent Deep Reinforcement Learning | ICML | 2021 | |
| Centralized Model and Exploration Policy for Multi-Agent | 2021 | ||
| REMAX: Relational Representation for Multi-Agent Exploration | AAMAS | 2022 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Model-based Multi-Agent Reinforcement Learning with Cooperative Prioritized Sweeping | 2020 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| MANAS: Multi-Agent Neural Architecture Search | 2019 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| MAMPS: Safe Multi-Agent Reinforcement Learning via Model Predictive Shielding | 2019 | ||
| Safer Deep RL with Shallow MCTS: A Case Study in Pommerman | 2019 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems | NIPS | 2021 | |
| Individual Reward Assisted Multi-Agent Reinforcement Learning | https://github.com/MDrW/ICML2022-IRAT | ICML | 2022 |
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| Networked Multi-Agent Reinforcement Learning in Continuous Spaces | IEEE conference on decision and control | 2018 | |
| Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learning | NIPS | 2019 | |
| Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents | ICML | 2018 |
For more adversarial / robust RL papers (single-agent and multi-agent), see Adversarial Reinforcement Learning Papers.
- TODO
Large Language Model (LLM) based multi-agent systems and the intersection of LLMs with (multi-agent) reinforcement learning. This is an emerging and fast-moving area.
| Paper | Code | Accepted at | Year |
|---|---|---|---|
| RPM: Generalizable Behaviors for Multi-Agent Reinforcement Learning | ICLR | 2023 | |
| BenchMARL: Benchmarking Multi-Agent Reinforcement Learning | https://github.com/facebookresearch/BenchMARL | JMLR | 2024 |
- Multi-Agent Path Finding
- Generalization in MARL
If you find this repository useful, please cite our repo:
@misc{chen2021multi,
author={Chen, Hao},
title={Multi-Agent Reinforcement Learning Papers},
year={2021}
publisher = {GitHub},
journal = {GitHub Repository},
howpublished = {\url{https://github.com/TimeBreaker/Multi-Agent-Reinforcement-Learning-papers}}
}