marl_mrt2a

KO | EN

gitlite — search

#llm #go #rust #deep-learning #docker #macos #nodejs #java #windows #awesome #cli #kubernetes #security #mcp

(★ 11)

No description available.

marl_mrt2a Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Learning Policies for Dynamic Coalition Formation in Multi-Robot Task Allocation

[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/release/python-380/)
[![IEEE Xplore](https://img.shields.io/badge/IEEE-Xplore-00629B.svg)](https://ieeexplore.ieee.org/document/11091462)
[![arXiv](https://img.shields.io/badge/arXiv-2412.20397-b31b1b.svg)](https://arxiv.org/abs/2412.20397)

This repository contains the code and simulation environments used in the paper:

**"Learning Policies for Dynamic Coalition Formation in Multi-Robot Task Allocation"**  
*Lucas C. D. Bezerra, Ataíde M. G. dos Santos, and Shinkyu Park*  
Accepted: IEEE Robotics and Automation Letters, 2025

## TL;DR:

This repository contains:

- **GyMRT²A Environment**: a Gym environment for discrete-space, discrete-time MRTA with multi-robot tasks.
- **MARL-MRT²A**: a MAPPO-based algorithm that enables learning of decentralized, low-communication, generalizable task allocation policies for MRT²A; this implementation builds upon the [EPyMARL](https://github.com/uoe-agents/epymarl) codebase.
- **PCFA Baseline**: our implementation of the decentralized market-based approach [PCFA](https://link.springer.com/article/10.1007/s10846-017-0493-x).

To get started, please see the [Installation](#installation) section and run the provided examples.

## Overview

We propose a decentralized, learning-based framework for **dynamic coalition formation** in Multi-Robot Task Allocation (MRTA) under **partial observability**. Our method extends MAPPO with multiple integrated components that allow robots to coordinate and revise task assignments in dynamic, partially observable environments.

### Key Components

- **Spatial Action Maps**: Agents select task locations in spatial coordinates, enabling long-horizon task planning.
- **Robot Motion Planning**: Each robot computes a collision-free A* path to the selected task.
- **Intention Sharing**: Robots share decayed path-based intention maps with nearby agents to support coordination.
- **Custom Policy Architecture**: We propose using a U-Net as the policy architecture, but our code supports custom architecture (that can be implemented as a `torch.nn.Sequential`; see [`nn_utils.py`](marl_mrt2a/marl/src/utils/nn_utils.py) for modules that are currently available)

## Environment

We implement our experiments in a custom Gym environment called **GyMRT²A**, which simulates:

- Grid-world task allocation with dynamic task spawns (Bernoulli or instant respawn)
- Partial observability (limited view and communication ranges)
- Multi-level tasks requiring varying coalition sizes
- Motion planning with obstacles and other agents

## Repository Structure

```
marl_mrt2a/
├── marl_mrt2a/
│   ├── env/               # GyMRT²A environment
│   ├── PCFA/              # Baseline implementation
│   ├── marl/              # Our method's implementation
│   └── examples/          # Reproducible experiments
│       └── main_comparison/    # Comparison with baseline and ablation studies
├── LICENSE
└── README.md
```

## Installation

### Prerequisites

- Python 3.10 or higher
- PyTorch
- NumPy
- OpenAI Gym

### Setup

1. Clone the repository:
```bash
git clone https://github.com/lcdbezerra/marl_mrt2a.git
cd marl_mrt2a
```

2. Create a conda environment:
```bash
conda create -n marl_mrt2a python=3.10 -y
conda activate marl_mrt2a
conda install pip -y
```

3. Install the base environment and the baseline (development mode)
```bash
cd marl_mrt2a/env
pip install -e .
pip install -U pygame --user
conda install -c conda-forge libstdcxx-ng -y
cd ../PCFA
pip install -e .
cd ../
```

4. Setup Weights & Biases for experiment tracking
```bash
pip install wandb
wandb login
```

5. Install MARL dependencies:
```bash
cd marl
pip install -r requirements.txt
```

## Experiments

Experiments are reproducible through the examples in the `examples/` directory:

### Main Comparison (`examples/main_comparison/`)
Compare the proposed method against baseline approaches:
- Traditional task allocation methods
- Standard MAPPO
- Other multi-agent learning approaches

<!-- ### 2. Generalizability (`examples/generalizability/`)
Test the framework's ability to generalize across:
- Different task types
- Varying environment configurations
- Diverse robot capabilities

### 3. Scalability (`examples/scalability/`)
Evaluate performance with:
- Different robot population sizes
- Varying task loads
- Large-scale scenarios

### 4. Visualization (`examples/watch_episode/`)
Monitor and visualize:
- Robot movements and coalitions
- Task allocation dynamics
- Learning progress -->

### Running Experiments

```bash
# Run main comparison experiments
python examples/main_comparison/run_comparison.py
```
<!-- ```bash
# Run main comparison experiments
python examples/main_comparison/run_comparison.py

# Run scalability experiments
python examples/scalability/run_scalability.py

# Run generalizability experiments
python examples/generalizability/run_generalizability.py

# Watch trained agents
python examples/watch_episode/watch_episode.py
``` -->

## Citation

If you use this code, please cite:

```bibtex
@article{bezerra2025learningdcfmrta,
  author={Lucas C. D. Bezerra and Ataíde M. G. dos Santos and Shinkyu Park},
  journal={IEEE Robotics and Automation Letters}, 
  title={Learning Policies for Dynamic Coalition Formation in Multi-Robot Task Allocation}, 
  year={2025},
  volume={10},
  number={9},
  pages={9216-9223},
  doi={10.1109/LRA.2025.3592080}}
```

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

For information about derivative works and third-party components, see the [NOTICE](NOTICE) file.

## Third-Party Code

This repository includes code under the Apache License 2.0:
- Multi-agent reinforcement learning framework based on [EPyMARL](https://github.com/uoe-agents/epymarl) and [PyMARL](https://github.com/oxwhirl/pymarl)
- `astar.py` – A* pathfinding implementation from [Red Blob Games](https://www.redblobgames.com/pathfinding/a-star/), Copyright 2014 Red Blob Games, licensed under Apache License 2.0. Adapted by Lucas C. D. Bezerra.

marl_mrt2a

marl_mrt2a Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

marl_mrt2a

marl_mrt2a Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README