Bskog AI

June 9, 2017

ConvNetJS – A JavaScript library for training Neural Networks in your browser

Here is a link to an Open Source JavaScript library that allows you to train neural networks in your browser. It was created by Andrej Karpathy, a PhD student at Stanford University and is currently community maintained.

It currently supports:

Common Neural Network modules (fully connected layers, non-linearities)
Classification (SVM/Softmax) and Regression (L2) cost functions
Ability to specify and train Convolutional Networks that process images
An experimental Reinforcement Learning module, based on Deep Q Learning.

Project site: http://cs.stanford.edu/people/karpathy/convnetjs/
Github: https://github.com/karpathy/convnetjs

June 9, 2017

Deep learning for Self driving cars

I found a great course for learning to create self driving cards using Deep learning, Deep Reinforcemet Learning, Convolutional Neural Networks and Recurrent Neural Network for different parts of the tasks needed to be solved in producing an autonomous vehicle that can adapt to traffic, control a car, learn to drive and steer through time.

Continue reading “Deep learning for Self driving cars”

June 8, 2017

Lasso regression

In order to avoid overfitting in regression due to too many feature while at the same time have enough features to minimize the sum of squared errors in order to get a more accurate fit on the test data, you need to regularize the regression.

This can be done with a Lasso regression where you want to minimize the sim of squared errors + plus a penalty parameters times the coefficient of the regression (which indicates the amount of features)

minimize SSE + λ|β|

June 1, 2017

Deep Learning School

Here you can watch lectures from the 2016 Deep Learning Summer School in Montreal.

Course excerpt:

Deep neural networks that learn to represent data in multiple layers of increasing abstraction have dramatically improved the state-of-the-art for speech recognition, object recognition, object detection, predicting the activity of drug molecules, and many other tasks. Deep learning discovers intricate structure in large datasets by building distributed representations, either via supervised, unsupervised or reinforcement learning.

The Deep Learning Summer School 2016 is aimed at graduate students and industrial engineers and researchers who already have some basic knowledge of machine learning (and possibly but not necessarily of deep learning) and wish to learn more about this rapidly growing field of research.

Here is the schedule in which you could view the presentations

table.schedule td {
vertical-align: top;
padding: 10px;
}

01/08/2016	02/08/2016	03/08/2016	04/08/2016	05/08/2016	06/08/2016	07/08/2016
9:00 10:30	Doina Precup Machine Learning	Rob Fergus Introduction to CNNs	Yoshua Bengio RNNs	Kyunghyun Cho Natural Language Understanding	Joelle Pineau Reinforcement Learning	Ruslan Salakhutdinov Deep Generative Models I	Bruno Olshausen Comp. Neuro I
10:30 11:00	Coffee Break	Coffee Break	Coffee Break	Coffee Break	Coffee Break	Coffee Break	Coffee Break
11:00 12:30	Hugo Larochelle Neural Networks I	Antonio Torralba Learning to See	Sumit Chopra Attention and Memory	Edward Grefenstette Beyond Seq2Seq	Pieter Abbeel Deep RL	Shakir Mohamed Building Machines that Imagine and Reason	Surya Ganguli Comp. Neuro II and Deep Learning Theory
12:30 14:30	Lunch	Lunch	LunchWiDL event	Lunch	Lunch	Lunch	Lunch
14:30 16:00	Hugo Larochelle Neural Networks II (click on part II)	Alex Wiltschko Torch I	Jeff Dean Large Scale Deep Learning & TensorFlow	Julie Bernauer (NVIDIA) GPU programming with CUDA	Joelle, Pieter & Doina Advanced Topics in RL	Contributed talks Session 4	Contributed talks Session 4
16:00 16:30	Coffee Break	Coffee Break	Coffee Break	Coffee Break	Coffee Break	Coffee Break	Coffee Break
16:30 18:00	Pascal Lamblin Theano I	Practical Session Alex Wiltschko (Torch) Frédéric Bastien (Theano)	Jeff Dean Large Scale Deep Learning & TensorFlow (click on part II)	Contributed talks Session 1	Contributed talks Session 2	Contributed Posters Session 1	Contributed Posters Session 2
Evening	Opening Reception (18:00-20:30) — by — Imagia	Happy Hour (18:45-22:30) buses at 18:30 — by — Maluuba	Happy Hour (18:30-20:30) — by — Creative Destruction Lab

(or you can just follow them in consecutive order at http://videolectures.net/deeplearning2016_montreal/ since they seem to be in the order they were presented.)

Contributed talks:

12:55 Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks
Rajarshi Das

14:29 Professor Forcing: A New Algorithm for Training Recurrent Networks
Anirudh Goyal

10:59 Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
Tegan Maharaj

18:58 Deep multi-view representation learning of brain responses to natural stimuli
Leila Wehbe

14:49 Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Jakob Foerster

13:57 Model-Based Relative Entropy Stochastic Search
Abbas Abdolmaleki

16:33 Learning Nash Equilibrium for General-Sum Markov Games from Batch Dat
Julien Pérolat

20:30 A Network-based End-to-End Trainable Task-oriented Dialogue System
Tsung-Hsien Wen

15:28 Inference Learning
Patrick Putzky

16:45 Variational Autoencoders with PixelCNN Decoders
Ishaan Gulrajani

13:33 An Infinite Restricted Boltzmann Machine
Marc-Alexandre Côté

15:15 Deep siamese neural network for prediction of long-range interactions in chromatin
Davide Chicco

14:09 Beam Search Message Passing in Bidirectional RNNs: Applications to Fill-in-the-Blank Image Captioning
Qing Sun

18:40 Analyzing the Behavior of Deep Visual Question Answering Models
Aishwarya Agrawal

13:55 Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation
Sina Honari

June 1, 2017

Introduction to Generative Adversarial Networks (GANs)

Read these articles:

and then read these papers by Ian Goodfellow et al:

You can also watch this video by Ian Goodfellow:

May 31, 2017

Outlier Rejection

To detect and get rid of outliers in a dataset (which may for instance have been caused by sensor error or data entry error) you first train your data, and remove the data point that has the highest residual error (over 10%) and then train again.

Otherwise erroneous data entries may give you an incorrect regression line.

May 31, 2017

Classification vs. Regression

Two slightly similar concepts in supervised machine learning are Supervised classification, and regression.
With supervised classification you will get a discrete output (a label or boolean value) and in regression your output is continuous (i.e. a number).
The thing you are trying to find in the different cases is a decision boundary when using classification and a best fit line in regression. You evalueate the former with it’s accuracy value, and the latter using the “sum of squared errors” or r².

May 25, 2017

Want to refresh your Linear Algebra?

In Machine Learning and especially Deep Learning you will need Linear Algebra. If you have not used your linear algebra in some time, this chapter of the Deep Learning Book will refresh you on the parts of Linear Algebra that are essential to machine Learning.

May 24, 2017

k-Nearest Neighbors

One of the simples algorithms in Machine Learning is k-Neares Neighbors. It is considered a “lazy learning” algorithm where all the calculations are deferred until classification.
It works like this:
You have out training data with its features and labels. Then upon classification, or testing the data, you take the k-nearest neighbors and assign the label for your test feature with what the majority vote of the k-nearest neighbors is. If k=1 then your features label with be that of its closes neighbor.

The Dataset