coordinated-ramp-metering-qmix

KO | EN

gitlite — search

#tailwindcss #self-hosted #http #agent #cloud-native #json #anthropic #claude #ui #gemini

coordinated-ramp-metering-qmix

(★ 9)

QMIX model, corridor network, and training framework for coordinated ramp metering in mixed-autonomy traffic

File Explorer

ZIP Download

- corridor.sumocfg
- my.net.xml
.gitignore
agent.py
config.py
qmix.py
qmix_net.py
ramp_env.py
README.md
replay_buffer.py
requirements.txt
train.py
worker.py

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This repository accompanies the paper

> H. Yu, L. Zhang, M. Zhang, F. Jin, and Y. Wang, “Coordinated ramp metering considering the dynamics of mixed-autonomy traffic,” *Sustainability*, vol. 16, no. 22, Art. no. 10055, Nov. 2024, doi: [10.3390/su162210055](https://doi.org/10.3390/su162210055).

The paper develops two coordinated ramp-metering methods for mixed HV/CAV traffic: **COE-HERO** (critical-occupancy-enhanced HERO) and **TRLCRM** (two-stage reinforcement-learning coordinated ramp metering). The code released here corresponds to the **TRLCRM** QMIX training stack.

It includes **three parts of the implementation**: the QMIX model, a SUMO corridor network, and the centralised-training decentralised-execution (CTDE) training framework. It is intended to make the method readable. **It is not a full reproduction package:** demand generation, observation construction, the reward, metering-rate mapping, vehicle-class parameters, and the training configuration used in the paper are omitted.

## What is included

```
config.py          # placeholder hyperparameters (not the paper settings)
qmix_net.py        # agent GRU and monotonic mixing network
qmix.py            # QMIX TD learner
agent.py           # epsilon-greedy action selection
replay_buffer.py   # episode replay
worker.py          # rollout protocol
ramp_env.py        # environment interface (methods not implemented)
train.py           # training-loop skeleton
sumo/my.net.xml    # freeway corridor with on-ramps
sumo/corridor.sumocfg
```

## Method (for reading the code)

Each on-ramp is an agent. At a decision step the agent receives a **local** observation and chooses a discrete metering action. A mixing network conditions on the **global** state and combines the agents' Q-values into a joint value \(Q_{\mathrm{tot}}\) that is trained with TD learning. Execution uses only the local Q-networks.

Local observations cover mainline occupancy and speed near the merge, short-term occupancy variation, on-ramp queue, and mixed-autonomy descriptors (CAV share and effective time gap). The feature list, normalisation, and reward are given in the paper, not in this repository.

`ramp_env.reset`, `step`, `get_obs`, and `get_state` raise `NotImplementedError` by design.

## Requirements

- Python 3.8+
- PyTorch
- NumPy
- Eclipse SUMO [3] if you inspect the `.net.xml` in Netedit / SUMO-GUI

```bash
pip install -r requirements.txt
```

Running `python train.py` only prints a notice; it does not start simulation or learning.

## References

The learning algorithm is QMIX [1]. The episode-buffer / CTDE training layout follows PyMARL [2]. Traffic simulation, when used with the released network file, is performed in SUMO [3].

[1] T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. Foerster, and S. Whiteson, “QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning,” in *Proc. 35th Int. Conf. Mach. Learn. (ICML)*, 2018, pp. 4295–4304.

[2] M. Samvelyan, T. Rashid, C. S. de Witt, G. Farquhar, N. Nardelli, T. G. J. Rudner, C.-M. Hung, P. H. S. Torr, J. Foerster, and S. Whiteson, “The StarCraft multi-agent challenge,” 2019, arXiv:1902.04043. Code: https://github.com/oxwhirl/pymarl

[3] P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wiessner, “Microscopic traffic simulation using SUMO,” in *Proc. 21st Int. Conf. Intell. Transp. Syst. (ITSC)*, 2018, pp. 2575–2582.

[4] H. Yu, L. Zhang, M. Zhang, F. Jin, and Y. Wang, “Coordinated ramp metering considering the dynamics of mixed-autonomy traffic,” *Sustainability*, vol. 16, no. 22, Art. no. 10055, Nov. 2024, doi: 10.3390/su162210055.

## Citation

Please cite [4] if you build on this code, and cite [1]–[3] for the algorithm, training framework, and simulator.

```bibtex
@article{yu2024coordinated,
  title   = {Coordinated Ramp Metering Considering the Dynamics of Mixed-Autonomy Traffic},
  author  = {Yu, Hongxin and Zhang, Lihui and Zhang, Meng and Jin, Fengyue and Wang, Yibing},
  journal = {Sustainability},
  volume  = {16},
  number  = {22},
  pages   = {10055},
  year    = {2024},
  doi     = {10.3390/su162210055}
}
```

coordinated-ramp-metering-qmix

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

coordinated-ramp-metering-qmix

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README