Jeremy Howard et al, at fast.ai has done what one might consider a huge breakthrough in regards to training deep learning models quickly.
They managed to train Imagenet in 18 minutes using publicly available resources that only cost them $40 to run!
this was their method:
- fast.ai’s progressive resizing for classification, and rectangular image validation
- NVIDIA’s NCCL with PyTorch’s all-reduce
- Tencent’s weight decay tuning; a variant of Google Brain’s dynamic batch sizes, gradual learning rate warm-up (Goyal et al 2018, and Leslie Smith 2018).
- ResNet-50 architecture
- SGD with momentum.
Here is a nice collection of Deep Learning resources including tutorials, papers and courses. Enjoy:
Stanford has released a dataset intended to be used to improve the state of the art in x-ray image classification.
Download it here and eter your own submission to the challenge: https://stanfordmlgroup.github.io/competitions/mura/
The large dataset for teaching your algorithms to drive can be downloaded from http://bdd-data.berkeley.edu/.
It contains over 100,000 HD video sequences, that make up over a thousand hours of footage. The data contains over 100 000 annotated images for object detection for bus, traffic light, traffic sign, person, bike, truck, motor, car, train, and rider. Alos segmentation, drivable area, lane markings etc.
I love how data is released to the public for the greater good.
The Fast and the Furious 2 of machine learning is now available for your pleasure.
Part 1, Part 2
Fast.ai is the very best way to learn practical Deep Learning. Period.
The first iteration of course 1 and 2, used Keras and the new versions use their own library built on top of PyTorch. Their new library is awesome and has a lot of useful best practice functions.
If you are interested in learning more about Data Science, you can check out the course page for the CS109 Data Science Course at Harvard University.
Topics covered are among others:
- Web Scraping
- Regular Expressions
- Data Reshaping
- Data Cleanup
- Frequentist Statistics
- Bias and Regression
- SVM, Decision Trees, Random Forests
- Ensemble Methods
- Bayes Theorem and Bayesian Methods
- Interactive Visualization
- Deep Networks
“Thanks everyone for an amazing month of January. It’s been an inspiring, life-changing experience for me.” – Lex Fridman
Several more lecture recordings are soon to be released.
Here is the official webpage of the course:
After a long wait, the final and much-anticipated course in the Coursera Deep Learning Specialization series taught by Andrew Ng, called Sequence Models, has now been released.
The first week will be about Recurrent Neural Networks, the second week will address Natural Language Processing & Word Embeddings and the final week will be about Sequence models & Attention mechanism.
Google colaboratory now has GPU support meaning you can run your jupyter notebooks on google drive with GPU support.
Here is a tutorial how to get started.
Last year the 2017 course of fast.ai was amazing, which taught state of the art deep learning to coders. There are so many goodies in the blog post about the Fast.ai 2018 launch which is available now. This year they held the course using Pytorch instead of Keras and wrote their own library for speeding up development and were the first to add several implementations from papers to the library such as Learning Rate Finder (Smith 2015) and Stochastic Gradient Descent with Restarts (SGDR). With one line of code, you can also get the images that the classifier gets wrong.
17 of the 20 top participants in a kaggle competitors were students in the preview course.
I recommend reading the blog post and taking the course.