sirius - GitLite

KO | EN

gitlite — search

#react #typescript #machine-learning #rust #nodejs #macos #java #windows #awesome #cli #awesome-list #vue #security #ios

sirius

(★ 67)

Official codebase for Sirius: Robot Learning on the Job

File Explorer

Download Latest Version (.zip)

- sirius.png
- - __init__.py
  - algo.py
  - awac.py
  - bc.py
  - bcq.py
  - cql.py
  - gl.py
  - hbc.py
  - iql.py
  - iris.py
  - td3_bc.py
- - __init__.py
  - awac_config.py
  - base_config.py
  - bc_config.py
  - bcq_config.py
  - config.py
  - cql_config.py
  - gl_config.py
  - hbc_config.py
  - iql_config.py
  - iris_config.py
  - td3_bc_config.py
  - vae_config.py
- - __init__.py
  - env_base.py
  - env_gym.py
  - env_ig_momart.py
  - env_robosuite.py
- - __init__.py
  - base_nets.py
  - distributions.py
  - obs_nets.py
  - policy_nets.py
  - vae_nets.py
  - value_nets.py
- - check_same_initial_configs.py
  - dataset_states_to_obs.py
  - download_datasets.py
  - download_momart_datasets.py
  - extract_obs_from_raw_datasets.sh
  - generate_config_templates.py
  - generate_paper_configs.py
  - get_dataset_info.py
  - hyperparam_helper.py
  - playback_dataset.py
  - run_trained_agent.py
  - split_train_val.py
  - train.py
- - __init__.py
  - dataset.py
  - env_utils.py
  - file_utils.py
  - hyperparam_utils.py
  - log_utils.py
  - loss_utils.py
  - macros.py
  - obs_utils.py
  - python_utils.py
  - tensor_utils.py
  - test_utils.py
  - torch_utils.py
  - train_utils.py
  - vis_utils.py
- __init__.py
LICENSE
MANIFEST.in
README.md
requirements-docs.txt
requirements.txt
setup.py
sirius.yml

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Sirius 🌟: Robot Learning on the Job

<br>

This is the official codebase for the [**Sirius**](https://ut-austin-rpl.github.io/sirius/) paper:

**Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment**
<br> [Huihan Liu](https://huihanl.github.io/), [Soroush Nasiriany](http://snasiriany.me/), [Lance Zhang](https://github.com/Lantian-Lance-Zhang), [Zhiyao Bao](https://www.linkedin.com/in/zhiyao-bao/), [Yuke Zhu](https://www.cs.utexas.edu/~yukez/) 
<br> [UT Austin Robot Perception and Learning Lab](https://rpl.cs.utexas.edu/)
<br> Robotics: Science and Systems (RSS), 2023
<br> **[[Paper]](https://arxiv.org/abs/2211.08416)** &nbsp;**[[Project Website]](https://ut-austin-rpl.github.io/sirius/)** &nbsp;**[[Real Robot Control]](https://ut-austin-rpl.github.io/deoxys-docs/html/getting_started/overview.html)**

<a href="https://ut-austin-rpl.github.io/sirius/" target="_blank"><img src="images/sirius.png" width="90%" /></a>

<br>

## Quickstart

Sirius builds upon [robomimic](https://github.com/ARISE-Initiative/robomimic), a framework for robot learning from demonstration. Sirius also uses the robotics simulator [robosuite](https://github.com/ARISE-Initiative/robosuite) powered by the MuJoCo physics engine.

### Setup Sirius codebase

#### Installing Sirius

```
git clone https://github.com/UT-Austin-RPL/sirius
cd sirius
conda env create -f sirius.yml
conda activate sirius
pip install -e .
```

#### Installing ```robosuite```

The additional reference for installing robosuite [here](https://robomimic.github.io/docs/introduction/installation.html) and [here](https://robosuite.ai/docs/installation.html#install-from-source) could be helpful.

```
$ git clone https://github.com/ARISE-Initiative/robosuite.git
$ cd robosuite
$ pip install -r requirements.txt
$ pip install -e .
```

## Usage

### Running Sirius

Running Sirius intervention-guided policy learning:

```
python robomimic/scripts/train.py --config robomimic/exps/sirius/sirius.json
```

IWR baseline:

```
python robomimic/scripts/train.py --config robomimic/exps/sirius/bc_iwr.json
```

BC baseline:

```
python robomimic/scripts/train.py --config robomimic/exps/sirius/bc.json
```

<br>

### Sirius Data Collection Pipeline

We include the script for collecting demonstrations and performing human intervention during robot policy execution below. We use a spacemouse for providing both demonstration and intervention. More details for setting up Spacemouse can be found [here](https://ut-austin-rpl.github.io/deoxys-docs/html/tutorials/using_teleoperation_devices.html).

#### Performing Human Demonstration

Perform human demonstration with the flag ```--all-demos```:

```
python robomimic/scripts/hitl/collect_hitl_demos.py --all-demos --num-traj 50
```

#### Policy Execution with Intervention

Perform human intervention with the policy checkpoint ```${checkpoint}```:

```
python robomimic/scripts/hitl/collect_hitl_demos.py --num-traj 50 --checkpoint ${checkpoint}
```

<br>

### Processing data

#### Adding modalities

By default, the datasets are generated in the minimum format with only low-level state information to save space. To add image observation and other modalities for training, run the following post-processing script. It will process the original data ```${data.hdf5}``` into ```${data_processed.hdf5}```, with image size ```${image_size}```. By default, the two camera view uses are agentview and robot0_eye_in_hand, which you can modify in the script ```template_process_sim_dataset.sh```.

```
cd robomimic/scripts/hitl

source template_process_sim_dataset.sh ${data.hdf5} ${data_processed.hdf5} ${image_size}
```

<br>

## Acknowledgements

This codebase is largely built on [robomimic](https://github.com/ARISE-Initiative/robomimic) and [robosuite](https://github.com/ARISE-Initiative/robosuite). We also thank [Ajay Mandlekar](https://ai.stanford.edu/~amandlek/) for sharing well-designed simulation task environments beyond the robomimic codebase like ```Coffee``` and ```Threading``` tasks during project development.

For real-robot experiments, we used [Deoxys](https://ut-austin-rpl.github.io/deoxys-docs/html/getting_started/overview.html), a controller library for Franka Emika Panda developed by [Yifeng Zhu](https://zhuyifengzju.github.io/).

<br>

## Citation
```bibtex
@inproceedings{liu2022robot,
    title = {Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment},
    author = {Huihan Liu and Soroush Nasiriany and Lance Zhang and Zhiyao Bao and Yuke Zhu},
    booktitle = {Robotics: Science and Systems (RSS)},
    year = {2023}
}
```