rainbow

KO | EN

gitlite — search

#gpt #developer-tools #list #nodejs #ui #privacy #android #bash #pdf #css

(★ 169)

A PyTorch implementation of Rainbow DQN agent

File Explorer

ZIP Download

- boxing.png
- breakout.png
- air_raid.bin
- alien.bin
- amidar.bin
- assault.bin
- asterix.bin
- asteroids.bin
- atlantis.bin
- bank_heist.bin
- battle_zone.bin
- beam_rider.bin
- berzerk.bin
- bowling.bin
- boxing.bin
- breakout.bin
- carnival.bin
- centipede.bin
- chopper_command.bin
- crazy_climber.bin
- defender.bin
- demon_attack.bin
- double_dunk.bin
- elevator_action.bin
- enduro.bin
- fishing_derby.bin
- freeway.bin
- frostbite.bin
- gopher.bin
- gravitar.bin
- hero.bin
- ice_hockey.bin
- jamesbond.bin
- journey_escape.bin
- kangaroo.bin
- krull.bin
- kung_fu_master.bin
- montezuma_revenge.bin
- ms_pacman.bin
- name_this_game.bin
- phoenix.bin
- pitfall.bin
- pong.bin
- pooyan.bin
- private_eye.bin
- qbert.bin
- riverraid.bin
- road_runner.bin
- robotank.bin
- seaquest.bin
- skiing.bin
- solaris.bin
- space_invaders.bin
- star_gunner.bin
- tennis.bin
- time_pilot.bin
- tutankham.bin
- up_n_down.bin
- venture.bin
- video_pinball.bin
- wizard_of_wor.bin
- yars_revenge.bin
- zaxxon.bin
.gitignore
core.py
dqn.py
env.py
logger.py
main.py
model.py
policy.py
README.md
train.py
utils.py

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This repo is a partial implementation of the [Rainbow](https://arxiv.org/pdf/1710.02298.pdf) 
agent published by researchers from DeepMind.
The implementation is efficient and of high quality. It trains at a speed of 
350 frames/s on a PC with a 3.5GHz CPU and GTX1080 GPU.

Rainbow is a deep Q learning based agent that combines a bunch of existing techiques
such as dueling dqn, distributional dqn, etc. This repo currenly implemented the 
following dqn variants:
* [DQN](https://www.nature.com/articles/nature14236)
* [Double DQN](https://arxiv.org/abs/1509.06461)
* [Dueling DQN](https://arxiv.org/abs/1511.06581)
* [Distributional DQN](https://arxiv.org/pdf/1707.06887.pdf)
* [Noisy Net](https://arxiv.org/abs/1706.10295)

and it will need the following extensions to become a full "Rainbow":
* Multi-step learning
* Priority Replay

## Hyperparameters

The hyperparameters in this repo follows the ones described in 
[Rainbow](https://arxiv.org/pdf/1710.02298.pdf)
paper as close as possible. However, there may still be some differences due to
misunderstanding.

## Performance

DQN agent often takes days to train. For sanity check, we can
train a agent to play a simple game "boxing". Follwing is the learning curve
of a dueling double dqn trained on boxing.

![](https://raw.githubusercontent.com/hengyuan-hu/rainbow/master/figs/boxing.png)

The agent almost solves boxing after around 12M frames, which is a good sign
that the implementation is working.

To test the distributional DQN and Noisy Net, the agent is trained on "breakout" since
distributional DQN performs significantly better than others on this game, 
reaching >400 scores rapidly while other DQN methods struggle to do so.

![](https://raw.githubusercontent.com/hengyuan-hu/rainbow/master/figs/breakout.png)

From the figure we see that the agent can reach >400 scores very rapidly and steadily.
Note that the publicly reported numbers on papers are produced by training the agent for
200M frames while here it trains only for 50M frames due to computation cost.

Figures here are smoothed.

## Future Works

We plan to implement multi-step learing and priority replay. Also, the current
implementation uses a simple wrapper on the [Arcade Learning Enviroment](https://github.com/mgbellemare/Arcade-Learning-Environment).
We may want to shift to OpenAI gym for better visualization and video recording.
On top of Rainbow, it will also be interesting to include other new techniques,
such as [Distributional RL with Quantile Regression](https://arxiv.org/pdf/1710.10044.pdf).

Contributions and bug-catchings are welcome!

rainbow

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

rainbow

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README