Bilevel-Coordinated-Reflection

KO | EN

gitlite — search

#electron #ui #developer-tools #large-language-models #svg #ai-agent #command-line #bash #git #automation

Bilevel-Coordinated-Reflection

(★ 9)

Official Code of Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems

File Explorer

ZIP Download

- bilevel.png
- __init__.py
- base.py
- local.py
- - __init__.py
  - _calibrate.py
  - _smoke_gate.py
  - _summ_gate.py
  - baselines.py
  - configs.py
  - plot.py
  - plot_gate_beta.py
  - run.py
  - run_gate.py
- - __init__.py
  - ABLATION_FINDINGS.md
  - configs.py
  - plot.py
  - plot_ablation.py
  - plot_gate_beta.py
  - RC_FORMULATION.md
  - run.py
  - run_ablation.py
- __init__.py
- __init__.py
- base.py
- markdown.py
- __init__.py
- llm.py
- orchestrator.py
- - __init__.py
  - adapter.py
  - common.py
  - config.py
  - driver.py
  - ground_truth.py
  - joint_astar.py
  - mock_adapter.py
  - original_adapter.py
  - planner.py
  - renderer.py
  - role.py
  - strategy.py
  - tools.py
- - __init__.py
  - baselines.py
  - driver.py
  - game.py
  - role.py
  - strategy.py
  - tools.py
- __init__.py
- __init__.py
- test_env_local.py
- test_memory.py
- test_oc_common.py
- test_oc_driver.py
- test_oc_mock.py
- test_oc_planner.py
- test_oc_renderer.py
- test_oc_strategy.py
- test_oc_tools.py
- test_orchestrator.py
- test_orchestrator_decompose.py
- test_orchestrator_rounds.py
- test_react.py
- test_react_hooks.py
- test_worker.py
- test_worker_agent_memory.py
- - __init__.py
  - fs.py
  - placeholders.py
  - shell.py
  - web.py
  - workboard.py
- __init__.py
- base.py
- hooks.py
- react.py
- retrieval.py
- role.py
- scratch.py
- tool.py
- toolsets.py
- worker.py
.gitignore
LICENSE
pyproject.toml
README.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Reference implementation for *Bilevel Coordinated Reflection: A Game-Theoretic Approach to
Multi-Agent LLM Systems* ([arXiv:2609.02750](https://arxiv.org/abs/2609.02750)). It
provides the bilevel orchestrator–worker framework and
**Stochastic Reflective Memory Ascent (SRMA)**, with two coordination testbeds:
**Resource Contest (RC)** and **Overcooked (OC)**.

> This release covers the RC and OC scenarios. The SWE-bench scenario will be open-sourced
> separately in about two weeks.

All agents are **frozen LLMs** called through an OpenAI-compatible endpoint; the only
adaptation is textual memory editing (no weights are trained or shipped). Every reported
metric is computed from environment ground truth — never from an LLM judge.

![Bilevel Coordinated Reflection](https://raw.githubusercontent.com/YihangChen9/Bilevel-Coordinated-Reflection/main/assets/bilevel.png)

*The upper-level orchestrator decomposes the task and evolves strategy memory `m_o` on the
slower timescale β; lower-level workers give η\_c-better responses and commit reflections to
the shared execution memory `m_e` only on verifier-risk descent, on the faster timescale α.*

---

## 1. What SRMA is

A candidate reflection is committed to shared memory **only when a deterministic, grounded
verifier risk `ρ` strictly decreases** (the gate). The verifier is scenario-specific:

| Scenario | Verifier risk `ρ` | Primary metric |
|---|---|---|
| Resource Contest | # workers with a concrete, environment-consistent cap claim | cumulative reward / regret |
| Overcooked | exact BFS value table `V(s)` = joint-action steps to next delivery | deliveries / score |

The upper level (orchestrator) decomposes the task and evolves a strategy memory across
episodes; the lower level (workers) read the full shared workboard, propose reflections,
and commit them only on `ρ`-descent.

---

## 2. Requirements

- Python **3.10–3.12**
- An OpenAI-compatible LLM endpoint (any provider; configured via `.env`, see §4)

---

## 3. Install

```bash
python -m venv .venv && source .venv/bin/activate

pip install -e .                    # core: Resource Contest + framework
pip install -e ".[overcooked]"      # Overcooked (uses overcooked_ai; pins numpy<2, scipy<1.13)
pip install -e ".[dev]"             # tests + plotting
```

> Overcooked pins `numpy<2` / `scipy<1.13` (required by `overcooked_ai`). If this clashes
> with other packages in your environment, use a dedicated virtualenv for `[overcooked]`.

---

## 4. Configure LLM access

Create a `.env` at the repo root (OpenAI-compatible endpoint):

```
LLM_API=openai
OPENAI_API_BASE=<your-endpoint>/v1
OPENAI_API_KEY=<your-key>
LLM_MODEL=<model-name>
LLM_MAX_TOKENS=2048
```

---

## 5. Quick verification

```bash
pytest tests/                                                 # unit tests
python -m experiments.oc.run --config cramped_room --no-llm   # OC baselines only, no API calls
```

---

## 6. Running the experiments

### Resource Contest
```bash
python -m experiments.rc.run --config easy            # one config
python -m experiments.rc.run_ablation --models <M> --variants algorithm_complete
```
Configs (`experiments/rc/configs.py`): `easy` (N=3, caps (3,5,8), T=15), `hard` (N=3,
(6,7,8), T=20), `many` (N=6, (2,4,5,6,7,9), T=20), `drift` (N=3, (3,5,8)→(9,5,4) at t=10).
Metric: cumulative reward / regret against the oracle `G*_t = max_i M_i`
(oracle Σ-reward 120 / 160 / 180 on easy/hard/many). The variant that fully realises the
algorithm is `algorithm_complete` (gated workboard writes + strategy reads workboard).

### Overcooked
```bash
python -m experiments.oc.run --config cramped_room    # bilevel + oracle/random baselines
python -m experiments.oc.run_gate --seeds 0 1 2       # gate vs. no-gate ablation
python -m experiments.oc._calibrate                   # drift-law β from gate logs
```
Layouts: `cramped_room`, `asymmetric_advantages`, `centre_pots`, plus hard-coordination
`coordination_ring`, `forced_coordination`, `counter_circuit`. Metric: deliveries / score.

---

## 7. Repository layout

```
worker/            ReAct agent, roles, tools, pre/post-step memory hooks
env/               Environment protocol + local stub
memory/            Markdown-backed memory ports (orch_log / workboard / agent scratch)
orchestrator/      llm.py (LiteLLM wrapper) + orchestrator.py (bilevel dispatcher)
scenarios/
  resource_contest/  game, roles, strategy, gated-reflection tools, baselines
  overcooked/        overcooked_ai adapter, BFS macro-planner, strategy, renderer
experiments/{rc,oc}/   runners, configs, ablations, plotting, calibration
tests/             unit tests
```

---

## 8. Notes

- LLMs are called through an OpenAI-compatible endpoint via LiteLLM; no weights are trained
  or shipped.
- Results, figures, and logs are regenerated by the commands above and are not checked in.
- See `LICENSE` for terms.

## Citation

If you use this code, please cite the paper:

```bibtex
@article{chen2026bilevel,
  title   = {Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems},
  author  = {Chen, Yihang and Chen, Yuxiang and Huang, Yuxuan and Fang, Meng and Luo, Weilin and Wang, Jun},
  journal = {arXiv preprint arXiv:2609.02750},
  year    = {2026},
}
```

Bilevel-Coordinated-Reflection

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Bilevel-Coordinated-Reflection

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README