The final Deep Learning Specialization course is now out

After a long wait, the final and much-anticipated course in the Coursera Deep Learning Specialization series taught by Andrew Ng, called Sequence Models, has now been released.

The first week will be about Recurrent Neural Networks, the second week will address Natural Language Processing & Word Embeddings and the final week will be about Sequence models & Attention mechanism.

 

Fast.ai 2018 has been released, and it’s truly awesome

Last year the 2017 course of fast.ai was amazing, which taught state of the art deep learning to coders. There are so many goodies in the blog post about the Fast.ai 2018 launch which is available now. This year they held the course using Pytorch instead of Keras and wrote their own library for speeding up development and were the first to add several implementations from papers to the library such as Learning Rate Finder (Smith 2015) and Stochastic Gradient Descent with Restarts (SGDR). With one line of code, you can also get the images that the classifier gets wrong.

17 of the 20 top participants in a kaggle competitors were students in the preview course.

I recommend reading the blog post and taking the course.

DeepSchool.io

There is a new kid on the block in terms of online courses on Deep Learning.

DeepSchool.io is a set of Jupyter notebooks that teach you the basics and different concepts you need in order to get started and being productive in Depp Learning. They are also videos supporting the notebook, although not for every notebook yet.

It differs from fast.ai in that the videos are shorter and the notebooks are mostly self-explanatory.

The goal of the project is to make Deep Learning accessible to everyone, make it practical, make learning open source and fun.

These are the topics covered:

  1. Lesson 0: Introduction to regression.
  2. Lesson 1: Penalising weights to fit better (scikit learn intro)

Mathematics (optional)

  1. Lesson 2: Gradient Descent. Using basic optimisation methods.
  2. Lesson 3: Tensorflow intro: zero layer hidden networks (i.e. normal regression).
  3. Lesson 4: Tensorflow hidden layer introduction.

Deep Learning

  1. Lesson 5: Using Keras to simplify multi-layer neural nets.
  2. Lesson 6: Embeddings to deal with categorical data. (Keras)
  3. Lesson 7: Word2Vec. Embeddings to visualise words. (Tensorflow)
  4. Lesson 8: Application – Bike Sharing predictions
  5. Lesson 9: Choosing Number of Layers and more
  6. Lesson 10: XGBoost – A quick detour from Deep Learning
  7. Lesson 11: Convolutional Neural Nets (MNIST dataset)
  8. Lesson 12: CNNs and BatchNormalisation (CIFAR10 dataset)
  9. Lesson 13: Transfer Learning (Dogs vs Cats dataset)

Advanced Topics

  1. Lesson 14: LSTMs – Sentiment analysis.
  2. Lesson 15: LSTMs – Shakespeare.
  3. Lesson 16: LSTMs – Trump Tweets.
  4. Lesson 17: Trump – Stacking and Stateful LSTMs.
  5. Lesson 18: Fake News Classifier

You can read more here.

Course 4 [deeplearning.ai] has been released!

The fourth course, Convolutional Neural Networks of Deeplearning.ai has now been released on coursera. People have been waiting for this one, but i think that the delay was to make the material very up to date with current research results. The four weeks of learning deals with:

  1. Foundations of Convolutional Neural Networks
  2. Deep convolutional models: case studies
  3. Object detection
  4. Special applications: Face recognition & Neural style transfer

Why fast.ai switched from Keras and Tensorflow to Pytorch and built their own Framework on top of it

In the new fast.ai course they will be using pytorch instead of Tensorflow, and has built a framework on top of it to make it even easier to use than Keras.

Pytorch is a dynamic instead of static deep learning library and Jeremy Writes that nearly all of the top 10 Kaggle competition winners now have been using Pytorch.

In the part 2 of fast.ai course the focus was to allow student so read and implement recent research papers, and pytorch made this easier due to its flexibility. It allowed them to try out things you could not do as easily with Tensorflow. It also makes it easier to understand what is going on in the algorithms as with Tensorflow, the computation becomes a black box once you send it to the GPU.

Most models trains faster on Pytorch than on Tensorflow and are easier to debug contributing to faster development iterations.

The reason they built a framework on top of Pytorch is that pytorch comes with less defaults than Keras. They want the course one to be accessible for students with little or no experience in Machine learning. Also they wanted to help avoid common pitfalls (such as not shuffling the data when needed to or vice versa) and get you going much faster, improving where Keras was lacking. They also built in many best practices that Keras was lacking. Jeremy writes that:

“We built models that are faster, more accurate, and more complex than those using Keras, yet were written with much less code.” – Jeremy Howard

The approach is to encapsulate all important data choices such as preprocessing, data augmentation, test/training/validation sets, multiclass/singleclass classification, regression and so on into Object-Oriented Classes.

“Suddenly, we were dramatically more productive, and made far less errors, because everything that could be automated, was automated.” – Jeremy Howard

Jeremy thinks that deep learning will see the same kind of library/framework explosion that front end developers have been used to during that last years. So the library you learn today will probably be obsolete in a year or two.

99,3% accuracy on dogs and cats with 3 lines of code is not bad:

Nvidia Deep Learning Institute

It is an awesome age we live in where the knowledge you need for tomorrow is available for free for everyone (with a computer, and an internet connection). There is more to learn than there is time to learn it in. We all can become experts in our fields. You must, however find places and situations to put your knowledge into practice so that it will not wane away. I think it is awesome that Nvidia has a learning institute with free courses to help you learn cutting edge stuff. By learning from a company focused on the advancing of the field, and who actually has only to gain from us learning us more, will keep you on the frontiers of the field.

https://developer.nvidia.com/dli/onlinelabs

Andrew Ng’s deeplearning.ai has released 5 deep learning specialization courses on Coursera

Andre Ng announced that he has launched five new courses in Deep Learning on Coursera.

The courses range from 2-4 weeks of study per course where you put in 3-6 hours of study per week per course.

  1. Neural Networks and Deep Learning
  2. Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
  3. Structuring Machine Learning Projects
  4. Convolutional Neural Networks
  5. Sequence Models

The courses will earn you a certificate and are described as follows:

In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. You will work on case studies from healthcare, autonomous driving, sign language reading, music generation, and natural language processing. You will master not only the theory, but also see how it is applied in industry. You will practice all these ideas in Python and in TensorFlow, which we will teach.