Some problems in the DQN example

[opocaj92]

Hi everyone,
First of all, thank you a lot, these exercises are being very useful for me to fix in mind what I've read on David Silver's slides.
I'm trying the DQN example, and I've found some errors. I'm referring to the dqn.py file in the repository for the line numbers:

1. Some attruibute in the TF summary creation (and also some other points somewhere else in the code) are wrong in the "skeleton" file, but are correct in the solution.
   These are:
   line 31: the size of the resized image should be [84, 84] (a list) and not 84, 84 (two separate attributes)
   line 62: tf.train.SummaryWriter should be tf.summary.FileWriter
   lines 106-110: tf.merge_summary should be tf.summary.merge, same for tf.scalar_summary and tf.histogram_summary with tf.summary.scalar and tf.summary.histogram respectively
   line 149: you use a class to copy the parameters between the two models in the solution, but a method in the skeleton (that's not important however, it works anyway)
   line 295: env.monitor.start() should be env=gym.wrappers.Monitor() (in the solution you use from gym.wrappers import Monitor at the beginning and then just env=Monitor(), that is the same). Also, line 381 should be removed (it is the call to env.monitor.close()).
2. line 377: the use of yield cause an error with the return statement "return with argument inside generator". This error is still there also in the solution, at least for me. If you comment the yield block, it works fine (but you're not getting the results, it just compute the solution).
3. Also, you should change the call to tf.initialize_all_variables() at line 401 (there is also another one somewhere in the skeleton file where you test if everything is setted up correctly) with tf.global_variables_initializer() (the method tf.initialize_all_variables() is deprecated and is going to be removed soon from the TF library).
   Hope it helps, thanks again for these fantastic exercises!