Google colaboratory now has GPU support meaning you can run your jupyter notebooks on google drive with GPU support.
Here is a tutorial how to get started.
Fast.ai 2018 has been released, and it’s truly awesome
Last year the 2017 course of fast.ai was amazing, which taught state of the art deep learning to coders. There are so many goodies in the blog post about the Fast.ai 2018 launch which is available now. This year they held the course using Pytorch instead of Keras and wrote their own library for speeding up development and were the first to add several implementations from papers to the library such as Learning Rate Finder (Smith 2015) and Stochastic Gradient Descent with Restarts (SGDR). With one line of code, you can also get the images that the classifier gets wrong.
17 of the 20 top participants in a kaggle competitors were students in the preview course.
I recommend reading the blog post and taking the course.
Tensorflow Object Detection API
Google has released an opensource framework built on top of Tensorflow, called the Tensorflow Object Detection API which is a tool for making it easy to make and deploy object detection models.
There are different state of the art types of models you can build. It you for instance make models using the Single Shot Multibox Detector (SSD) with MobileNets you will get lightweight models that you can run in real time on mobile devices.
The models you get are Single Shot Multiboc Detector, using MobileNets or Inception V2, RegionBased Fully Convolutional Networks with Resnet 101, and Faster RCMM with Resnet 101 or Inception Resnet v2.
You also get a Jupyter notebook for trying things out
If all these terms above makes no sense, you can read this excellent blog post explaining Deep Learning for Object Detection by Joyce Xu.
Why fast.ai switched from Keras and Tensorflow to Pytorch and built their own Framework on top of it
In the new fast.ai course they will be using pytorch instead of Tensorflow, and has built a framework on top of it to make it even easier to use than Keras.
Pytorch is a dynamic instead of static deep learning library and Jeremy Writes that nearly all of the top 10 Kaggle competition winners now have been using Pytorch.
In the part 2 of fast.ai course the focus was to allow student so read and implement recent research papers, and pytorch made this easier due to its flexibility. It allowed them to try out things you could not do as easily with Tensorflow. It also makes it easier to understand what is going on in the algorithms as with Tensorflow, the computation becomes a black box once you send it to the GPU.
Most models trains faster on Pytorch than on Tensorflow and are easier to debug contributing to faster development iterations.
The reason they built a framework on top of Pytorch is that pytorch comes with less defaults than Keras. They want the course one to be accessible for students with little or no experience in Machine learning. Also they wanted to help avoid common pitfalls (such as not shuffling the data when needed to or vice versa) and get you going much faster, improving where Keras was lacking. They also built in many best practices that Keras was lacking. Jeremy writes that:
“We built models that are faster, more accurate, and more complex than those using Keras, yet were written with much less code.” – Jeremy Howard
The approach is to encapsulate all important data choices such as preprocessing, data augmentation, test/training/validation sets, multiclass/singleclass classification, regression and so on into Object-Oriented Classes.
“Suddenly, we were dramatically more productive, and made far less errors, because everything that could be automated, was automated.” – Jeremy Howard
Jeremy thinks that deep learning will see the same kind of library/framework explosion that front end developers have been used to during that last years. So the library you learn today will probably be obsolete in a year or two.
99,3% accuracy on dogs and cats with 3 lines of code is not bad:
ConvNetJS – A JavaScript library for training Neural Networks in your browser
Here is a link to an Open Source JavaScript library that allows you to train neural networks in your browser. It was created by Andrej Karpathy, a PhD student at Stanford University and is currently community maintained.
It currently supports:
- Common Neural Network modules (fully connected layers, non-linearities)
- Classification (SVM/Softmax) and Regression (L2) cost functions
- Ability to specify and train Convolutional Networks that process images
- An experimental Reinforcement Learning module, based on Deep Q Learning.
Project site: http://cs.stanford.edu/people/karpathy/convnetjs/
Github: https://github.com/karpathy/convnetjs