rl-mcp

KO | EN

gitlite — search

#csharp #containers #mysql #ai-agent #gemini #json #open-source #video #docker #cross-platform

(★ 9)

Wrap rl training env to make it a mcp-server.

파일 탐색기

ZIP 다운로드

- settings.json
- - ci.yml
  - publish.yml
- - isaaclab-anymal-shot.png
  - records-story.png
  - records-tree.png
  - reorient-final-policy.gif
- curriculum.md
- design.md
- extensions.md
- genesis.md
- isaaclab.md
- mcp-server.md
- records.md
- style.md
- tools.md
- tuning.md
- your-task.md
- - genesis_env.py
  - go1_cfg.py
  - README.md
  - train_genesis.py
  - train_mjlab.py
- train_g1_rough_curriculum.py
- train_isaaclab.py
- - __init__.py
  - base.py
  - env_wrapper.py
  - reward_terms.py
  - rsl_rl_runner.py
- - __init__.py
  - controller.py
  - curriculum.py
  - diagnostics.py
  - extensions.py
  - live_view.py
  - palette.py
  - progress_video.py
  - replay.py
  - reward_source.py
- - __init__.py
  - registry.py
  - terrain.py
- - __init__.py
  - clips.py
  - filestore.py
  - graph.py
  - importer.py
  - interventions.py
  - link.py
  - params.py
  - poster.py
  - recipe.py
  - record.py
  - report.py
  - snapshot.py
  - store.py
  - validate.py
  - views.py
- - __init__.py
  - mcp_server.py
- __init__.py
- check.py
- cli.py
- cli_output.py
- env_export.py
- hostd.py
- play.py
- py.typed
- registry.py
- rewards_export.py
- session.py
- source_bundle.py
- tasks.py
- train.py
- wire.py
- conftest.py
- test_access.py
- test_backend_conformance.py
- test_backend_isolation.py
- test_check.py
- test_cli_discovery.py
- test_cli_dispatch.py
- test_cli_output.py
- test_clips.py
- test_controller.py
- test_curriculum.py
- test_diagnostics.py
- test_env_export.py
- test_env_wrapper.py
- test_examples.py
- test_extensions.py
- test_flat_env_access.py
- test_flat_env_state.py
- test_flat_reward_terms.py
- test_genesis_adapter.py
- test_graph.py
- test_interventions.py
- test_isaaclab_adapter.py
- test_live_replay.py
- test_live_view.py
- test_mcp_optional.py
- test_mcp_tools.py
- test_palette.py
- test_parameter_registry.py
- test_params.py
- test_play.py
- test_poster.py
- test_progress_video.py
- test_public_surface.py
- test_recipe.py
- test_record.py
- test_records_link.py
- test_records_store.py
- test_records_validate.py
- test_registry.py
- test_replay.py
- test_reward_lifecycle.py
- test_reward_terms.py
- test_reward_weights.py
- test_session.py
- test_snapshot.py
- test_tasks.py
- test_telemetry.py
- test_train_recipe_flags.py
- test_wire.py
.git-blame-ignore-revs
.gitignore
.pre-commit-config.yaml
AGENTS.md
CLAUDE.md
LICENSE
pyproject.toml
README.md

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

**Watch, diagnose and steer a live RL training run — from an AI agent, or from
your shell.**

Training a policy is a slow feedback loop. Launch, wait an hour, squint at
TensorBoard, guess which reward weight was wrong, relaunch. rlmcp shortens it.

Attach it to a running job and you can ask the live process what the robot is
doing, look at it, measure how jerky it is, change a reward weight, unlock harder
terrain, and roll back if that made things worse. No restart.

Two backends ship: [mjlab](https://github.com/mujocolab/mjlab), which is what
this is developed against, and [IsaacLab](https://github.com/isaac-sim/IsaacLab).
The simulator sits behind a small adapter, so a third is a couple of hundred
lines rather than a fork.

> ⚠️ Experimental. The surface is real and tested, but it is still moving.

---

## 🚀 Start here

**If you use a coding agent**, give it one line:

```text
Wrap my mjlab env with rlmcp (github.com/mktk1117/rl-mcp) and show me how to use it.
```

(Say IsaacLab instead if that is what you train on — the docs cover both.)

It reads the docs here, adds the wrapper to your training script, and drives the
run from there.

**If you are driving yourself**, jump to [Install](#install) and
[Try it](#try-it).

---

## What it looks like

```bash
rlmcp status                                   # iteration, stage, headline metrics
rlmcp diagnose --seconds 4                     # is the gait smooth? is it tracking?
rlmcp shot --where terrain=pyramid_stairs      # look at a robot on the stairs
rlmcp video --seconds 5                        # and one at 0, 50, 100, 200 … unasked
rlmcp view                                     # where the browser view is (a run has one)
rlmcp set reward.action_rate_l2.weight -0.25 --why "ankles chattering at 15 Hz"
rlmcp run set_terrain terrains='["flat","random_rough"]' max_level=4
rlmcp curriculum advance --why "flat is solved"
rlmcp checkpoint before-experiment             # and `rlmcp load` to undo
rlmcp events --interventions                   # everything anyone did, and why
rlmcp play                                     # watch a finished run's policy
```

Almost all of that is an MCP tool too, so an agent can drive the whole loop,
and **it sees the screenshots and plots**, not just their file paths. `play` is
the exception, and stays CLI-only because it opens a window.

The CLI knows who is reading. A terminal gets aligned tables, and pictures open
by themselves. A pipe gets JSON, and each command's JSON is a parsing contract
that does not move — the shapes are written down and pinned by tests, in
[docs/tools.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/tools.md#what-a-pipe-gets).

## The whole integration

One call, between building the environment and building the runner.

```python
from mjlab.envs import ManagerBasedRlEnv
import rlmcp

env = ManagerBasedRlEnv(cfg=env_cfg, device="cuda:0", render_mode="rgb_array")
env = rlmcp.wrap(env, session_dir=log_dir / "rlmcp", curriculum="terrain")

vec_env = RslRlVecEnvWrapper(env)
runner  = VelocityOnPolicyRunner(vec_env, agent_cfg, str(log_dir), device)
env.attach_runner(runner)          # PPO knobs, checkpoints, iteration boundaries

runner.learn(num_learning_iterations=agent_cfg.max_iterations)
```

That is it. Details and traps: [docs/your-task.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/your-task.md).

### The same one call on IsaacLab

IsaacLab launches its own app before anything else can be imported, so the wrap
goes after `gym.make`. Downstream is the same CLI, the same MCP tools and the
same JSON, minus what only mjlab can answer — the terrain commands, which an
IsaacLab run reports as absent rather than broken.

```python
env = gym.make(task, cfg=env_cfg, render_mode="rgb_array")
env = rlmcp_isaaclab.wrap(env, session_dir=log_dir / "rlmcp", task_id=task,
                          service_every_steps=agent_cfg.num_steps_per_env)
```

`rlmcp shot --env-id 300`, taken mid-training from a stock
`Isaac-Velocity-Rough-Anymal-D-v0` run: one robot on the stairs, with the other
2047 environments carrying on behind it. Nothing about the task was described to
rlmcp — 73 tunable knobs came out of walking the environment's own manager
configs, the same walk it uses on mjlab. The example script is
[examples/train_isaaclab.py](examples/train_isaaclab.py), and the things that
differ — cameras, the robot's name, which Isaac Sim goes with which driver — are
in [docs/isaaclab.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/isaaclab.md).

## What you get

| | |
| --- | --- |
| **Look at the robot** | Screenshots and clips of real training steps, not a separate eval. Pick which robot with `--where terrain=stairs`. |
| **Watch it live** | Every run serves a browser view; `rlmcp view` says where. It needs no renderer, so it works on a headless box, and it costs nothing while nobody has the tab open — which is why it is on rather than a flag. The tab has mjlab's own panels: reward bars, term plots, contact and force overlays. **Pause view** (or `rlmcp view --pause`) freezes the tab and hands the run back its full speed, without giving the port back. `--realtime` buffers a window and plays it at the speed the robot actually moves, with mjlab's player in the tab. |
| **Know why it moves badly** | `diagnose` measures jerk, chatter, effort, posture and gait, then says which lever to pull. When it cannot measure properly, it says so instead of guessing. |
| **Tune anything, live** | 97 knobs on the G1 rough task, discovered from the environment. Reward weights, randomization ranges, PPO hyperparameters. Applied between rollout batches, never mid-step. |
| **A ladder that drives itself** | Curriculum stages promote on earned conditions, not a schedule. Override any of it from a shell. |
| **Undo** | Checkpoints save weights *and* parameters, stage and extension state. |
| **Your task's own words** | Extensions add verbs, metrics and env selectors. They reach the CLI, MCP and curriculum stages at once, without editing rlmcp. |
| **A record that survives** | Every change is logged with its reason, and every run stamps the code it launched with. Runs are kept as a graph with hypotheses and verdicts, so three failures in a row become one conclusion. |
| **Look at a finished run** | `rlmcp play` restores the conditions a checkpoint trained under before rendering it. Otherwise a good policy looks broken. |

## A worked example: in-hand cube reorientation

A SharpaWave hand turning a cube to a commanded orientation anywhere in SO(3),
then holding it there for a full second. This is the final policy, replayed from
its checkpoint with `rlmcp play`:

Ten runs to get there. Three worked, four were falsified, three were stopped
early. That is the normal shape of RL work, which is why rlmcp keeps runs as a
record instead of a folder of logs. `rlmcp record graph` writes one
self-contained HTML page — no server, no build step — and it looks like this:

Each card is one run: the clip it produced, and the one line it concluded. Click
any of them and the panel on the right shows what that run predicted, what would
have proved it wrong, what changed since its parent, and what it measured.

Runs 005 to 007 are the interesting part. The policy froze, because standing
still paid about 2600x more than making progress. Cheap drops broke the freeze
and produced the mirror failure: the cube thrown off the palm within half a
second. Re-pricing the trade in between failed too. Three failures, one
conclusion: the reward was never the binding constraint. 008 changed the action
interface instead — an EMA filter — and went from 1.1 to 16.8 goals per minute
with the cube held 99.4% of the time.

That conclusion is not in any single run. It is in the shape of three siblings,
which is the argument for keeping runs as a graph.

The same page has a **tree** view for the ancestry itself and a **parameters**
view for what changed across runs. More in [docs/records.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/records.md).

## Install

```bash
pip install rl-mcp              # training side: numpy, matplotlib, pillow, imageio
pip install 'rl-mcp[server]'    # adds the MCP SDK for the agent side
```

`uv pip install` takes the same lines. For what is on `main` right now, or to
skip installing altogether and run the CLI straight from GitHub:

```bash
pip install 'rl-mcp[server] @ git+https://github.com/mktk1117/rl-mcp'
uvx --from 'rl-mcp[server] @ git+https://github.com/mktk1117/rl-mcp' rlmcp sessions
```

A checkout installs as `pip install -e '.[server]'`.

The MCP SDK is optional on purpose. The training process does not need it, and
the server does not need a simulator. Both `mcp>=2` and `mcp` 1.x work.

The simulator is yours to install. rlmcp asks only that it be importable from
the training process — for IsaacLab that means installing into the interpreter
Isaac Sim runs under.

```bash
claude mcp add rlmcp -- rlmcp-server --root /path/to/logs
```

## Try it

```bash
rlmcp tasks         # which ids exist here, before anything has run
# train, with the terrain ladder driving itself
rlmcp-train Mjlab-Velocity-Rough-Unitree-G1 --num-envs 4096
```

Then, from another shell:

```bash
rlmcp sessions      # what is running
rlmcp status
rlmcp curriculum
rlmcp diagnose --seconds 4
```

The explicit-curriculum version of the same run is in
[examples/train_g1_rough_curriculum.py](examples/train_g1_rough_curriculum.py).
On IsaacLab, the equivalent first run is
[examples/train_isaaclab.py](examples/train_isaaclab.py) — same four commands
afterwards.

## Documentation

| page | for |
| --- | --- |
| [docs/tools.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/tools.md) | **Every tool, one entry each.** Shell command, MCP call, what comes back, and the traps. Start here for either audience. |
| [docs/mcp-server.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/mcp-server.md) | Agents: setup, session pinning, liveness, deferred jobs, a worked steering session. |
| [docs/tuning.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/tuning.md) | The tuning loop: verify the task first, watch the right numbers, read symptoms into levers. The distilled findings of two campaigns. |
| [docs/your-task.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/your-task.md) | Putting rlmcp on your own task, in five steps. |
| [docs/curriculum.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/curriculum.md) | Writing the stage ladder. |
| [docs/extensions.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/extensions.md) | Teaching rlmcp your task's vocabulary. |
| [docs/isaaclab.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/isaaclab.md) | Driving an IsaacLab run: the one line, cameras, what differs from mjlab. |
| [docs/records.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/records.md) | Hypotheses, verdicts, feedback, code snapshots, the record graph. |
| [docs/design.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/design.md) | How it fits together, how parameters are found, other simulators. |
| [docs/style.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/style.md) | The style guide: two spaces, what ruff checks, and why each rule is on or off. |
| [AGENTS.md](https://github.com/mktk1117/rl-mcp/blob/main/AGENTS.md) | Contributing to this repository. |

## How it works, briefly

Two processes that never touch each other. They talk through a plain directory
of JSON files.

```
   training process                session directory                 agent process
  torch + a simulator               plain JSON files                  no simulator
┌───────────────────┐             ┌─────────────────┐             ┌──────────────────┐
│ rlmcp-wrapped env │             │ status.json     │             │ MCP server       │
│ publishes metrics │ ──writes──► │ metrics.jsonl   │ ───reads──► │ (38 tools)       │
│ records frames    │             │ events.jsonl    │             │ rlmcp CLI        │
│ runs commands     │ ◄──reads─── │ artifacts/      │ ◄──writes── │ your own scripts │
│ between batches   │             │ inbox/  outbox/ │             │                  │
└───────────────────┘             └─────────────────┘             └──────────────────┘
```

Telemetry flows left to right. Commands flow right to left. Nobody shares
memory, and nothing blocks the training loop.

That buys four things:

- The agent side never imports torch or a simulator, so starting or killing it
  cannot disturb training.
- Commands run **between rollout batches**, so an edit can never race the
  simulator.
- Everything is inspectable with `cat` when something goes wrong.
- A run leaves a complete, replayable record behind after it exits.

The same split is why a second backend was cheap. Parameter discovery, trace
sampling, metrics and the wrapper are written against the shape mjlab and
IsaacLab share; each adapter is only what is genuinely its own — how a robot is
found in the scene and how a frame is rendered. More in
[docs/design.md](https://github.com/mktk1117/rl-mcp/blob/main/docs/design.md).

## Tests

```bash
pytest tests -q     # 855 tests, ~10s, no GPU and no simulator required
```

## License

Apache-2.0 — see [LICENSE](LICENSE).

rl-mcp

오너의 다른 레포

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

rl-mcp

오너의 다른 레포

파일 탐색기

# CDN으로 사용하기

Was this content helpful?

Recent Feedback

Download README