rainbow - GitLite

KO | EN

gitlite — search

#large-language-models #list #go #containers #markdown #rust #machine-learning #linux #typescript #automation

rainbow

(★ 169)

A PyTorch implementation of Rainbow DQN agent

파일 탐색기

ZIP 다운로드

- boxing.png
- breakout.png
- air_raid.bin
- alien.bin
- amidar.bin
- assault.bin
- asterix.bin
- asteroids.bin
- atlantis.bin
- bank_heist.bin
- battle_zone.bin
- beam_rider.bin
- berzerk.bin
- bowling.bin
- boxing.bin
- breakout.bin
- carnival.bin
- centipede.bin
- chopper_command.bin
- crazy_climber.bin
- defender.bin
- demon_attack.bin
- double_dunk.bin
- elevator_action.bin
- enduro.bin
- fishing_derby.bin
- freeway.bin
- frostbite.bin
- gopher.bin
- gravitar.bin
- hero.bin
- ice_hockey.bin
- jamesbond.bin
- journey_escape.bin
- kangaroo.bin
- krull.bin
- kung_fu_master.bin
- montezuma_revenge.bin
- ms_pacman.bin
- name_this_game.bin
- phoenix.bin
- pitfall.bin
- pong.bin
- pooyan.bin
- private_eye.bin
- qbert.bin
- riverraid.bin
- road_runner.bin
- robotank.bin
- seaquest.bin
- skiing.bin
- solaris.bin
- space_invaders.bin
- star_gunner.bin
- tennis.bin
- time_pilot.bin
- tutankham.bin
- up_n_down.bin
- venture.bin
- video_pinball.bin
- wizard_of_wor.bin
- yars_revenge.bin
- zaxxon.bin
.gitignore
core.py
dqn.py
env.py
logger.py
main.py
model.py
policy.py
README.md
train.py
utils.py

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This repo is a partial implementation of the [Rainbow](https://arxiv.org/pdf/1710.02298.pdf) 
agent published by researchers from DeepMind.
The implementation is efficient and of high quality. It trains at a speed of 
350 frames/s on a PC with a 3.5GHz CPU and GTX1080 GPU.

Rainbow is a deep Q learning based agent that combines a bunch of existing techiques
such as dueling dqn, distributional dqn, etc. This repo currenly implemented the 
following dqn variants:
* [DQN](https://www.nature.com/articles/nature14236)
* [Double DQN](https://arxiv.org/abs/1509.06461)
* [Dueling DQN](https://arxiv.org/abs/1511.06581)
* [Distributional DQN](https://arxiv.org/pdf/1707.06887.pdf)
* [Noisy Net](https://arxiv.org/abs/1706.10295)

and it will need the following extensions to become a full "Rainbow":
* Multi-step learning
* Priority Replay

## Hyperparameters

The hyperparameters in this repo follows the ones described in 
[Rainbow](https://arxiv.org/pdf/1710.02298.pdf)
paper as close as possible. However, there may still be some differences due to
misunderstanding.

## Performance

DQN agent often takes days to train. For sanity check, we can
train a agent to play a simple game "boxing". Follwing is the learning curve
of a dueling double dqn trained on boxing.

![](https://raw.githubusercontent.com/hengyuan-hu/rainbow/master/figs/boxing.png)

The agent almost solves boxing after around 12M frames, which is a good sign
that the implementation is working.

To test the distributional DQN and Noisy Net, the agent is trained on "breakout" since
distributional DQN performs significantly better than others on this game, 
reaching >400 scores rapidly while other DQN methods struggle to do so.

![](https://raw.githubusercontent.com/hengyuan-hu/rainbow/master/figs/breakout.png)

From the figure we see that the agent can reach >400 scores very rapidly and steadily.
Note that the publicly reported numbers on papers are produced by training the agent for
200M frames while here it trains only for 50M frames due to computation cost.

Figures here are smoothed.

## Future Works

We plan to implement multi-step learing and priority replay. Also, the current
implementation uses a simple wrapper on the [Arcade Learning Enviroment](https://github.com/mgbellemare/Arcade-Learning-Environment).
We may want to shift to OpenAI gym for better visualization and video recording.
On top of Rainbow, it will also be interesting to include other new techniques,
such as [Distributional RL with Quantile Regression](https://arxiv.org/pdf/1710.10044.pdf).

Contributions and bug-catchings are welcome!