deep-rl-toolkit

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #golang #android #rust #deep-learning #linux #macos #awesome #cli #vue #security #openai

deep-rl-toolkit

(★ 9)

RLToolkit is a flexible and high-efficient reinforcement learning framework. Include implementation of DQN, AC,A2C, A3C, PG, DDPG, TRPO, PPO, SAC, TD3 and ....

deep-rl-toolkit Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Deep-RL-Toolkit

## Overview

Deep RL Toolkit is a flexible and high-efficient reinforcement learning framework. RLToolkit is developed for practitioners with the following advantages:

- **Reproducible**. We provide algorithms that stably reproduce the result of many influential reinforcement learning algorithms.

- **Extensible**. Build new algorithms quickly by inheriting the abstract class in the framework.

- **Reusable**.  Algorithms provided in the repository could be directly adapted to a new task by defining a forward network and training mechanism will be built automatically.

- **Elastic**: allows to elastically and automatically allocate computing resources on the cloud.

- **Lightweight**: the core codes \<1,000 lines (check [Demo](examples/cleanrl/cleanrl_runner.py)).

- **Stable**: much more stable than [Stable Baselines 3](https://github.com/DLR-RM/stable-baselines3) by utilizing various ensemble methods.

## Table of Content

- [Deep-RL-Toolkit](#deep-rl-toolkit)
  - [Overview](#overview)
  - [Table of Content](#table-of-content)
  - [Supported Algorithms](#supported-algorithms)
  - [Supported Envs](#supported-envs)
  - [Examples](#examples)
    - [Quick Start](#quick-start)
  - [References](#references)
    - [Reference Papers](#reference-papers)
    - [References code](#references-code)

## Supported Algorithms

RLToolkit implements the following model-free deep reinforcement learning (DRL) algorithms:

![../_images/rl_algorithms_9_15.svg](https://spinningup.openai.com/en/latest/_images/rl_algorithms_9_15.svg)

## Supported Envs

- **OpenAI Gym**
- **Atari**
- **MuJoCo**
- **PyBullet**

For the details of DRL algorithms, please check out the educational webpage [OpenAI Spinning Up](https://spinningup.openai.com/en/latest/).

## Examples

If you want to learn more about deep reinforcemnet learning, please read the [deep-rl-class](https://jianzhnie.github.io/llmtech/) and run the [examples](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples).

- [Classic Control](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/discrete)
- [Atari Benchmark](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/atari)
- [Box2d Benchmark](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/box2d)
- [Mujuco Benchmark](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/mujoco)
- [Petting Zoo](https://github.com/jianzhnie/deep-rl-toolkit/blob/main/examples/pettingzoo)

### Quick Start

```bash
git clone https://github.com/jianzhnie/deep-rl-toolkit.git

# Run the DQN algorithm on the CartPole-v0 environment
python examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo dqn
python examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo ddqn
python examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo dueling_dqn
python examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo dueling_ddqn

# Run the C51 algorithm on the CartPole-v0 environment
python examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo c51

# Run the DDPG algorithm on the Pendulum-v1 environment
python examples/cleanrl/cleanrl_runner.py --env Pendulum-v0 --algo ddpg

# Run the PPO algorithm on the CartPole-v0 environment
python examples/cleanrl/cleanrl_runner.py --env CartPole-v0 --algo ppo
```

## References

### Reference Papers

01. Deep Q-Network (DQN) ([V. Mnih et al. 2015](https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf)) 
02. Double DQN (DDQN) ([H. Van Hasselt et al. 2015](https://arxiv.org/abs/1509.06461)) 
03. Advantage Actor Critic (A2C)
04. Vanilla Policy Gradient (VPG)
05. Natural Policy Gradient (NPG) ([S. Kakade et al. 2002](http://papers.nips.cc/paper/2073-a-natural-policy-gradient.pdf)) 
06. Trust Region Policy Optimization (TRPO) ([J. Schulman et al. 2015](https://arxiv.org/abs/1502.05477)) 
07. Proximal Policy Optimization (PPO) ([J. Schulman et al. 2017](https://arxiv.org/abs/1707.06347)) 
08. Deep Deterministic Policy Gradient (DDPG) ([T. Lillicrap et al. 2015](https://arxiv.org/abs/1509.02971)) 
09. Twin Delayed DDPG (TD3) ([S. Fujimoto et al. 2018](https://arxiv.org/abs/1802.09477)) 
10. Soft Actor-Critic (SAC) ([T. Haarnoja et al. 2018](https://arxiv.org/abs/1801.01290)) 
11. SAC with automatic entropy adjustment (SAC-AEA) ([T. Haarnoja et al. 2018](https://arxiv.org/abs/1812.05905))

### References code

- rllib

- https://github.com/ray-project/ray
  - https://docs.ray.io/en/latest/rllib/index.html

- coach

- https://github.com/IntelLabs/coach
  - https://intellabs.github.io/coach

- Pearl

- https://github.com/facebookresearch/Pearl
  - https://pearlagent.github.io/

- tianshou

- https://github.com/thu-ml/tianshou
  - https://tianshou.org/en/stable/

- stable-baselines3

- https://github.com/DLR-RM/stable-baselines3
  - https://stable-baselines3.readthedocs.io/en/master/

- PARL

- https://github.com/PaddlePaddle/PARL
  - https://parl.readthedocs.io/zh-cn/latest/

- openrl

- https://github.com/OpenRL-Lab/openrl/
  - https://openrl-docs.readthedocs.io/zh/latest/

- cleanrl

- https://github.com/vwxyzjn/cleanrl
  - https://docs.cleanrl.dev/

deep-rl-toolkit

deep-rl-toolkit Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

deep-rl-toolkit

deep-rl-toolkit Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README