cloak - GitLite

KO | EN

gitlite — search

#self-hosted #hacktoberfest #automation #cpp #kotlin #cli #ai-agents #command-line #large-language-models #chatgpt

cloak

(★ 9)

No description available.

File Explorer

ZIP Download

- - close-stale-issues.yml
- - 2f85.xml
  - fr3_robotiq.xml
  - scene.xml
- - fr3_sharpa_angled.xml
  - right_sharpa_ha4_v2_1.xml
  - scene.xml
- - fr3_umi.xml
  - panda_hand_umi.xml
  - scene.xml
- - scene.xml
  - yam_linear.xml
- droid_intrinsics_processed.json
- droid_lang_annotations_processed.json
- droid_wrist_extrinsics.json
- droid_zed_serials.json
- - __init__.py
  - calibration_utils.py
- - __init__.py
- - __init__.py
  - parameters.py
  - server_interface.py
  - time.py
- __init__.py
- deploy_policy.py
- franka_sharpa_env.py
- robot_env.py
- serve_policy.py
- - apple-touch-icon.png
  - favicon-16x16.png
  - favicon-32x32.png
  - favicon.ico
  - main_result_avg.svg
  - overview.svg
  - teaser.png
- index.html
- README.md
- render_extrinsics.py
- compute_norm_stats.py
- preprocess_data.py
- preprocess_metadata.py
- preprocess_wrist_extrinsics.py
- deploy_policy.sh
- serve_policy.sh
- train.sh
- train_debug.sh
- - __init__.py
  - constants.py
  - py.typed
  - transforms.py
- - __init__.py
  - base_policy.py
  - msgpack_numpy.py
  - msgpack_numpy_test.py
  - websocket_client_policy.py
- train.py
- utils.py
.gitignore
LICENSE
pyproject.toml
README.md
README_train.md
uv.lock

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

# Project Badges

Project Page Visit

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

A framework for robot policy calibration, training, and deployment.

## Installation

```bash
uv sync
```

This installs the base (policy-serving) dependencies. The robot client and
training loop pull in their own extras automatically via `uv run --group
client ...` / `uv run --group train ...` (used by `deploy_policy.sh` /
`train.sh`) — no separate install step needed.

(Optional) Download trained checkpoints from [here](https://drive.google.com/drive/folders/1DWXAGzNjsc-iyNWnbEMEgyhw4JkP-a7f?usp=drive_link). The checkpoint name corresponds to the train config in `src/openpi/training/config.py`.

## Training

Please see `README_train.md` for training details.

## Deployment

Deployment has two halves connected over a websocket: a **policy server** (GPU workstation) and a **robot client**.

The robot client requires the deployment hardware:
- **Franka arm + ZED camera** — the standard DROID robot platform. See the [DROID setup docs](https://droid-dataset.github.io/droid).
- **Sharpa hand** (for Sharpa configs) — obtain the `SharpaWaveSDK_4.3.4/` SDK and place it at the repo root; it is loaded at runtime. Requires Python 3.10–3.12.

Serve the policy:

```bash
bash scripts/serve_policy.sh
```

Set `CONFIG`, `CHECKPOINT_DIR`, and `EMBODIMENT` at the top of the script. One config
(`pi05_full_droid_finetune_v3-1_ik`) serves every robot; `EMBODIMENT` (`robotiq` | `sharpa` | `umi` | `yam`) selects
the wrist mask renderer and cross-embodiment IK at serve time.

Run the client:

```bash
bash scripts/deploy_policy.sh
```

Set `EMBODIMENT` (matching the server's) and your ZED serials `EXTERNAL_CAMERA_ID` / `WRIST_CAMERA_ID` (`SN<serial>`
files under `/usr/local/zed/settings/`) at the top of `deploy_policy.sh`. Set the `CLOAK_NUC_IP` env var to the Franka
NUC's IP; if unset, the controller runs locally.

## Calibrated DROID wrist extrinsics

We release per-episode wrist-camera extrinsics for [DROID](https://droid-dataset.github.io/) episodes, the camera's
pose relative to the end-effector. Each is recovered by our **Silhouette Calibration** algorithm, which produces more
accurate alignment to sim than the extrinsics shipped with DROID.

- `assets/droid_wrist_extrinsics.json` — JSON containing the 6-DoF camera pose relative to the Franka attachment_site.
- `examples/render_extrinsics.py` — Minimal example of using the extrinsics to render the wrist view.

To regenerate them via Silhouette Calibration, see [Data preprocessing](README_train.md) (optional — we ship the precomputed extrinsics).