distributedRL - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #machine-learning #rust #docker #nodejs #awesome #cli #pytorch #mcp #ios #openai #open-source

distributedRL

(★ 97)

A framework for easy prototyping of distributed reinforcement learning algorithms

파일 탐색기

최종 버전 다운로드 (.zip)

- __init__.py
- config.yml
- dqn_learner.py
- dqn_worker.py
- models.py
- run_apex_dqn.py
- __init__.py
- apex.py
- - __init__.py
  - architecture.py
  - learner.py
  - worker.py
- - __init__.py
  - baseline_wrappers.py
  - buffer.py
  - buffer_helper.py
  - segtree.py
  - utils.py
.gitignore
__init__.py
environment.yml
LICENSE
README.md
setup.py

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/cyoon1729/distributedRL

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd distributedRL

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -e .

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# distributedRL

distributedRL is a framework for prototyping disrtibuted reinforcement learning using **Pytorch**, **Ray**, and **ZeroMQ** (and more). You can make use of the structural components to easily distribute your reinforcement learning algorithm, on a single-machine.

Currently, an implementation of *Ape-X DQN* is available. *IMPALA* and *SEED* are planned to be implemented.

Feel free to reach out (cjy2129 at columbia dot edu) or raise an issue if you have any questions!

### A note about a few choices 
- `ApeXLearner` and `ApeXWorker` are implemented as abstract classes (that inherit `common.abstract.Learner` and `common.abstract.Worker` respectively). To extend Ape-X to any off-policy RL algorithm, you just have to implement algorithm-specific details, such as action selection, environment step, and learning step (check the abstract classes for more detail). 
- I use [ZeroMQ](https://zeromq.org/) for inter-process communication instead of the Ray's built-in features. (1) I wanted the data-passing mechanism and serialization to be a bit more explicit, and (2) ray's inter-process communication forbids calling remote operations, which I found to be a bit restricting for this purpose.

### Installation
clone the repository, then
```
conda env create -f environment.yml
conda activate distrl
pip install -e .
```

### A Short Tutorial
*To be added*

### Benchmarking
*To be added*

### Acknowledgements
I thank [@Avrech](https://github.com/avrech) for his helpful discussion, suggestions and enhancements. In particular, on identifying bugs and bottlenecks and improving asynchrony of the Ape-X implementation.

### References/Papers:
- [Distributed Prioritized Experience Replay (Horgan et al., 2018), ICLR 2018](https://arxiv.org/abs/1803.00933)