Here is a good tutorial explaining how convolutions work:

http://deeplearning.net/software/theano_versions/dev/tutorial/conv_arithmetic.html

Here is an example of a convolution with half (one) padding and stride 2

Skip to content
# Category: Learning material

## Convolutional Arithmetic tutorial

## How To Learn Fast

x1 x2 x2.5 x3 x3.25 x3.5 x4

## The maths you will need for AI/Machine Learning

## Nvidia Deep Learning Institute

## Fast AI course on Computational Linear Algebra

## Andrew Ng’s deeplearning.ai has released 5 deep learning specialization courses on Coursera

## Tips on how to pick up Deep Learning

## Yet another free deep learning online course

## Another Deep Learning School

### Introduction to Feedforward Neural Networks

### Deep Learning for Computer Vision

### Deep Learning for NLP

### Foundations of Deep Unsupervised Learning

### Deep Reinforement Learning

### Theano Tutorial

### Deep Learning for Speech

### Torch Tutorial

### Sequence to Sequence Learning for NLP and Speech

### Foundations and Challenges of Deep Learning

## The best free introductory course to machine learning according to the Internet

AI, Machine Learning and Deep Learning Blog

The field of Machine Learning is rapidly advancing. Here i try to provide you with the resources you need in order to climb the mountain of knowledge in AI and Deep Learning so that you will be able to contribute yourself to this field.

Here is a good tutorial explaining how convolutions work:

http://deeplearning.net/software/theano_versions/dev/tutorial/conv_arithmetic.html

Here is an example of a convolution with half (one) padding and stride 2

In two days i was able to listen through half of cs231n in my spare time by listening on the youtube videos with higher than normal speed.

Nowadays i always listen to youtube videos with 2x or 3x speed.

With normal settings you can st the speed up to 2x. If you want to get the video faster than that you need to add a plugin or bookmarklet to achieve that.

You can drag these liks to your bookmarks bar, and get them as speed buttons to adjust the speed of your youtube videos…

x1 x2 x2.5 x3 x3.25 x3.5 x4

Also, check out this video on how to learn advanced concepts fast:

This Hacker News thread discusses why and what kind of maths you will need if you pursue AI/Machine learning.

Here is a short summary, and i tend to agree. These where mandatory maths courses when i studied CS :

You need to have a solid foundation in:

Good to know:

- Graph theory or Discrete math. (no course on khan academy for that, but on great courses, which isn’t free)

Here are some books:

- “Information Theory, Inference and Learning Algorithms” by David MacKaye.
- “Probability Theory: the Logic of Science” by E. T. Jaynes.
- “Elements of Statistical Learning” by Tibshirani is also good.
- “Bayesian Data Analysis” by Andrew Gelman is another great read.
- “Deep Learning” by Ian Goodfellow and Yoshua Bengio

I like the following quote motivating why you for instance will need calculus:

*Calculus essentially discusses how things change smoothly and it has a very nice mechanism for talking about smooth changes algebraically.*

* A system which is at an optimum will, at that exact point, be no longer increasing or decreasing: a metal sheet balanced at the peak of a hill rests flat.*

* Many problems in ML are optimization problems: given some set of constraints, what choices of unknown parameters minimizes error? This can be very hard (NP-hard) in general, but if you design your situation to be “smooth” then you can use calculus and its very nice set of algebraic solutions. – Commend by used Tel*

It could bee very motivating for students when they first start with calculus, linear algebra and statistics if they have an idea in what fields they can practically use them later on.

It is an awesome age we live in where the knowledge you need for tomorrow is available for free for everyone (with a computer, and an internet connection). There is more to learn than there is time to learn it in. We all can become experts in our fields. You must, however find places and situations to put your knowledge into practice so that it will not wane away. I think it is awesome that Nvidia has a learning institute with free courses to help you learn cutting edge stuff. By learning from a company focused on the advancing of the field, and who actually has only to gain from us learning us more, will keep you on the frontiers of the field.

You need to put in 20% of your learning time into math in order to get great at machine learning. Linear algebra and statistics are two very imprtant topics to cover. Here is a fast.ai course on Computational Linear Algebra taught in a different way. It is very hands on. You will be programming.

Andre Ng announced that he has launched five new courses in Deep Learning on Coursera.

The courses range from 2-4 weeks of study per course where you put in 3-6 hours of study per week per course.

- Neural Networks and Deep Learning
- Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
- Structuring Machine Learning Projects
- Convolutional Neural Networks
- Sequence Models

The courses will earn you a certificate and are described as follows:

In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. You will work on case studies from healthcare, autonomous driving, sign language reading, music generation, and natural language processing. You will master not only the theory, but also see how it is applied in industry. You will practice all these ideas in Python and in TensorFlow, which we will teach.

Here is a Quora thread on some of the resources you can use to get Deep Learning skills.

https://www.quora.com/What-are-the-best-ways-to-pick-up-Deep-Learning-skills-as-an-engineer

If you do not know what tasks to start to tackle you can try one of these https://openai.com/requests-for-research/, enter a kaggle competition or participate in http://course.fast.ai and do the assignments there.

This course is held one of the contributors to Keras and Tensoflow.

Here you go:

https://campus.datacamp.com/courses/deep-learning-in-python

I recently posted about Deep Learning Summer School 2016

Here is a link to another Bay Area Deep Learning School one that was the same year in September at Stanford CA.

*Hugo Larochelle*

I will cover some of the fundamental concepts behind feedforward neural networks. I’ll start by briefly reviewing the basic multi-layer architecture of feedforward networks, as well as backpropagation from automatic differentiation and stochastic gradient descent (SGD). Then, I’ll discuss the most recent ideas that are now commonly used for training deep neural networks, such as variants of SGD, dropout, batch normalization and unsupervised pretraining.

*Andrej Karpathy*

I will cover the design of convolutional neural network (ConvNet) architectures for image understanding, the history of state of the art models on the ImageNet Large Scale Visual Recognition Challenge, and some of the most recent patterns of developments in this area. I will also talk about ConvNet architectures in the context of related visual recognition tasks such as object detection, segmentation, and video processing.

*Richard Socher*

I will describe the foundations of deep learning for natural language processing: word vectors, recurrent neural networks, tasks and models influenced by linguistics. I will end with some recent models that put together all these basic lego blocks into a very powerful deep architecture called dynamic memory network.

*Ruslan Salakhutdinov*

Building intelligent systems that are capable of extracting meaningful

representations from high-dimensional data lies at the core of solving many Artificial Intelligence tasks, including visual object recognition, information retrieval, speech perception, and language understanding. In this tutorial I will discuss mathematical basics of many popular unsupervised models, including Sparse Coding, Autoencoders, Restricted Boltzmann Machines (RBMs), Deep Boltzmann Machines (DBMs), and Variational Autoencoders (VAE). I will furtherdemonstrate that these models are capable of extracting useful hierarchical representations from high dimensional data with applications in visual object recognition, information retrieval, and natural language processing. Finally, time permitting, I will briefly discuss models that can generate natural language descriptions (captions) of images, as well as generate images from captions using attention mechanism.

*John Schulman, OpenAI*

I’ll start by providing an overview of the state of the art in deep reinforcement learning, including recent applications to video games (e.g., Atari), board games (AlphaGo) and simulated robotics. Then I’ll give a tutorial introduction to the two methods that lie at the core of these results: policy gradients and Q-learning. Finally, I’ll present a new analysis that shows the close similarity between these two methods. A theme of the talk will be to not only ask “what works?”, but also “when does it work?” and “why does it work?”; and to find the kind of answers that are actionable for tuning one’s implementation and designing better algorithms.

*Pascal Lamblin*

Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently, on CPU or GPU. Since its introduction, Theano has been one of the most popular frameworks in the machine learning community, and multiple frameworks for deep learning have been built on top of it (Lasagne, Keras, Blocks, …). This tutorial will focus first on the concepts behind Theano and how to build and evaluate simple expressions, and then we will see how more complex models can be defined and trained.

*Adam Coates*

Traditional speech recognition systems are built from numerous modules, each requiring its own challenging engineering. With deep learning it is now possible to create neural networks that perform most of the tasks of a traditional engine “end to end”, dramatically simplifying the development of new speech systems and opening a path to human-level performance. In this tutorial, we will walk through the steps for constructing one type of end-to-end system similar to Baidu’s “Deep Speech” model. We will put all of the pieces together to form a “scale model” of a state of the art speech system; small-scale versions of the neural networks now powering production speech engines.

*Alex Wiltschko*

Torch is an open platform for scientific computing in the Lua language, with a focus on machine learning, in particular deep learning. Torch is distinguished from other array libraries by having first-class support for GPU computation, and a clear, interactive and imperative style. Further, through the “NN” library, Torch has broad support for building and training neural networks by composing primitive blocks or layers together in compute graphs. Torch, although benefitting from

extensive industry support, is a community owned and community developed ecosystem. All neural net libraries, including Torch NN, TensorFlow and Theano, rely on automatic differentiation (AD) to manage the computation of gradients of complex compositions of functions. I will present some general background on automatic differentiation (AD), which is the fundamental abstraction of gradient based optimization, and demonstrate

Twitter’s flexible implementation of AD in the library torch-autograd

*Quoc Le*

I will first present the foundations of sequence to sequence (seq2seq) learning and attention models, and their applications in machine translation and speech recognition. Then I will discuss attention with pointers and functions. Finally I will describe how reinforcement learning can play a role in seq2seq and attention models.

*Yoshua Bengio*

Why is deep learning working as well as it does? What are some big challenges that remain ahead? This talk will first survey some key factors in the success of deep learning. First, from the context of the no-free lunch theorem, we will discuss the expressive power of deep netwroks to capture abstract distributed representations. Second, we will discuss our surprising ability to actually optimize the parameters of neural networks in spite of their non-convexity. We will then consider a few challenges ahead, including the core representation question of disentangling the underlying explanatory factors of variation, especially with unsupervised learning, why this is important for bringing reinforcement learning to the next level, and optimization questions that remain challenging, such as learning of long-term dependencies, understanding the optimization landscape of deep networks, and how learning in brains remain a mystery worth attacking from the deep learning perspective.

Many people have recommended the Machine Learning course held by Andrew Ng at Stanford University that is available via Coursera. Some say it is the best course they have heard. You can participate for free if you want, but if you want a certificate that you have taken the course you can buy the course for a small fee.