[ExI] Breakout success for reinforcement learning

[Alejandro Dubrovsky]

On 28/02/15 22:35, Anders Sandberg wrote:
> Google DeepMind's paper on reinforcement learning playing Atari games is
> now out in Nature: Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A.,
> Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari
> with deep reinforcement learning. arXiv preprint arXiv:1312.5602.
>
> http://googleresearch.blogspot.co.uk/2015/02/from-pixels-to-actions-human-level.html
> http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html
> https://www.youtube.com/watch?v=iqXKQf2BOSE
> http://arxiv.org/abs/1312.5602
>

Yep, DeepMind are killing it at the moment. They've also released their 
source code with their Nature paper. You can get it from here: 
https://sites.google.com/a/deepmind.com/dqn/

There's also code independently developed by Nathan Sprague based on the 
2013 version of the paper here: https://github.com/spragunr/deep_q_rl. 
The 2013 version used a narrower and shallower network, and it also 
didn't use their new two-network system that they adapted in the Nature 
version to avoid oscillatory behaviour. At the moment the only advantage 
of that code would be the more liberal licencing since DeepMind's code 
can only be used for research purposes. It shouldn't take long for it to 
catch up with the newer version, and I suspect it will become the go-to 
version because of the licencing advantage.

BTW, if you are going to play with the code, make sure to get the 
beastliest Nvidia GPU that you can afford.