Paper on Deep Reinforcement Learning

Paper: A Brief Survey of Deep Reinforcement Learning
Authors: Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, Anil Anthony Bharath
Submitted: Submitted on 19 Aug 2017
Read the PDF


Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world. Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to play video games directly from pixels. Deep reinforcement learning algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of reinforcement learning, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep reinforcement learning, including the deep Q-network, trust region policy optimisation, and asynchronous advantage actor-critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via reinforcement learning. To conclude, we describe several current areas of research within the field.

Nvidia Deep Learning Institute

It is an awesome age we live in where the knowledge you need for tomorrow is available for free for everyone (with a computer, and an internet connection). There is more to learn than there is time to learn it in. We all can become experts in our fields. You must, however find places and situations to put your knowledge into practice so that it will not wane away. I think it is awesome that Nvidia has a learning institute with free courses to help you learn cutting edge stuff. By learning from a company focused on the advancing of the field, and who actually has only to gain from us learning us more, will keep you on the frontiers of the field.

Andrew Ng’s has released 5 deep learning specialization courses on Coursera

Andre Ng announced that he has launched five new courses in Deep Learning on Coursera.

The courses range from 2-4 weeks of study per course where you put in 3-6 hours of study per week per course.

  1. Neural Networks and Deep Learning
  2. Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
  3. Structuring Machine Learning Projects
  4. Convolutional Neural Networks
  5. Sequence Models

The courses will earn you a certificate and are described as follows:

In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. You will work on case studies from healthcare, autonomous driving, sign language reading, music generation, and natural language processing. You will master not only the theory, but also see how it is applied in industry. You will practice all these ideas in Python and in TensorFlow, which we will teach.

Practical Deep Learning for coders (

This is in my opinion the best free course on getting into the state of the art in deep learning. It is a site that offers a free 7 week learning experience for deep learning. taught by 2 year in a row Kaggle winner, entrepreneur and generally nice guy Jeremy Howard and Math PhD/Data scientist/Full stack developer/Forbes Featured Rachel Thomas, two amazing people in AI. Their approach to teaching Deep Learning for Coders is that it shall be accessible to as many people as possible and not to a selected few. So instead of abstract mathy lectures, they allow you to get your hands dirty from the first lecture and improve your intuition of the field, thus enabling you to create state of the art deep learning solutions from day one.

After starting the course, i immediately realized that these are very talented educators that are sincere about their goal to make AI accessible to everyone, and to make it benefit others. What i especially like about the course is the way they approach the topic pedagogically. Their method is inspired by the book “Making Learning Whole: How Seven Principles of Teaching can Transform Education” by author David Perkins. Perkins compares todays education with learning baseball:

If you would learn baseball the way that math is taught, you would first learn about the shape of a parabola, and then you would learn the material science behind the stitchings in baseballs and so forth. And twenty years later after you have completed your PhD and post-doc, you would be taken to your first baseball game and you would be introduced to the rules of baseball. And then 10 years later you might get to hit. The way that in practice baseball is taught is we take the kid down to the baseball diamond and we say “These people are playing baseball, would you like to play?” And they say, “Yeah! Sure I would”. “Perfect, stand here, i’m gonna throw this. Hit it. Ok, great, now run. Good you’re playing baseball”

That is why the first class of the course they demonstrate that here are 7 lines of code that you can use to perform state of the art image classification using deep learning. And to do any image classification you want as long as you structure it the right way. You may not understand most of it, but as you need to adapt the tasks to your needs, you will need to learn more details, and thus you learn.

The course consists of a 2 hour lecture each week, detailed lecture notes, a community contributed wiki and jupyter notebooks which you also will do your assignments in. (There is also setup instructions for getting a GPU equipped machine up and running on AWS) In The first weeks assigment you will submit an entry into the Kaggle competition for classifying cat and dog images. By taking advantage of what you learn you will outperform what was the state of the art when the competition was launched 2013.