(Originally posted on Quora as an answer to How do I learn deep learning in 2 months?)
If you have coding experience with an engineering background or relevant knowledge in mathematics and computer science, in just two months you can become proficient in deep learning. Hard to believe? Here's a four-step process that makes it possible.
For more inspiration check out the following video by Andrew Ng
(Optional, but highly recommended)
Start with Andrew Ng's Class on machine learning https://www.coursera.org/learn/machine-learning. His course provides an introduction to the various Machine Learning algorithms currently out there and, more importantly, the general procedures and methods for machine learning, including data preprocessing, hyper-parameter tuning, and more.
I would also recommend reading the NIPS 2015 Deep Learning Tutorial by Geoff Hinton, Yoshua Bengio, and Yann LeCun, which offers an introduction at a slightly lower level.
My personal learning preference is to watch lecture videos, and there are several excellent courses online. Here are few classes I especially like and can recommend:
- Deep Learning at Oxford 2015 Taught by Nando de Freitas who expertly explains the basics, without overcomplicating it. Start with Lecture 9 if you are already familiar with Neural Networks and want to go deep. He uses Torch framework in his examples. (Videos on Youtube)
- Neural Networks for Machine Learning: Geoffrey Hinton’s class on Coursera. Hinton is an excellent researcher who demonstrated the use of generalized backpropagation algorithm and was crucial to the development of deep learning. I have utmost respect for him, but I found the delivery of this course bit unorganized. Furthermore, coursera messes up with the placement of quizzes. Still worth a look, though.
- Neural Networks Class by Hugo Larochelle: Another excellent course
- Yaser Abu-Mostafa's Machine Learning Course: More theory, if you are interested.
If you are more into books as a primary learning tool, here are some excellent resources.
- Neural Networks and Deep Learning Book by Michael Nielsen: Online book that offers several interactive JavaScript elements to play with.
- Deep Learning Book by Ian Goodfellow, Yoshua Bengio and Aaron Courville: A bit denser, but never the less a great resource
The next step is to identify what you are passionate about and go deeper. The field is vast, so this list is in no way a comprehensive one.
Deep Learning has transformed this area. Stanford’s CS231N course by Andrej Karpathy's course is the best course I have come across; CS231n Convolutional Neural Networks for Visual Recognition. It teaches you the basics up to convnets, as well as helping you to set up GPU instance in AWS. Also, take the time to check out Getting Started in Computer Vision by Mostafa S. Ibrahim
Used for machine translation, question and answering and sentiment analysis. To master this field, an in-depth understanding of both algorithms and the underlying computational properties of natural languages is a must. CS 224N / Ling 284 by Christopher Manning is a great course to get started. CS224d: Deep Learning for Natural Language Processing, another Stanford class by David Socher (founder of MetaMind)is an excellent course to progress to, as it goes over all the latest Deep Learning research related to NLP. For more details see How do I learn Natural Language Processing?
Recent work in combining attention mechanism in LSTM Recurrent Neural networks with external writable memory has led to some impressive work in building systems that can understand, store and retrieve information in a question & answering style. This research area got its start in Dr. Yann Lecun’s Facebook AI lab at NYU. The original paper is available on arxiv: Memory Networks. There are then a number of research variants, datasets, benchmarks, etc. that have stemmed from this work to aid further learning. For example, Metamind's Dynamic Memory Networks for Natural Language Processing is a great resource
Deep Reinforcement Learning was made famous by AlphaGo, the Go-playing system that recently defeated the strongest Go players in history. David Silver's (Google Deepmind) Video Lectures on RL, and Professor Rich Stutton's book are a great place to start. For a gentler introduction to LSTM see Christopher’s post on Understanding LSTM networks and Andrej Karpathy’s The Unreasonable Effectiveness of Recurrent Neural Networks
While discriminatory models try to detect, identify and separate things, they end up looking for features which differentiate and do not understand data at a fundamental level. Apart from the short-term applications, generative models provide the potential to automatically learn natural features; categories or dimensions or something else entirely. Out of the three commonly used generative models — Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs) and Autoregressive models (such as PixelRNN), GAN's are most popular. To dig deeper read
- Original GAN paper by Ian Goodfellow et al.
- The Laplacian Adversarial Networks (LAPGAN) Paper (LAPGAN) which fixed the stability issue
- The Deep Convolutional Generative Adversarial Networks (DCGAN) paper and DCGAN Code which can be used to learn a hierarchy of features without any supervision. Also, check out DCGAN used for Image Superresolution
Reading and watching lessons is great, but doing is the real key to becoming an expert. Try to create something which interests you and matches your skill level. Here are a few suggestions to get you thinking:
- As is tradition, start with classifying the MNIST dataset
- Try face detection and classification on ImageNet. If you are up to it, do the ImageNet Challenge 2016.
- Perform a Twitter sentiment analysis using RNNs or CNNs
- Teach neural networks to reproduce the artistic style of famous painters (A Neural Algorithm of Artistic Style)
- Compose Music With Recurrent Neural Networks
- Play ping-pong using Deep Reinforcement Learning
- Use Neural Networks to Rate a selfie
- Automatically color Black & White pictures using Deep Learning
For more inspiration, take a look at CS231n Winter 2016 and Winter 2015 projects. Also, keep an eye on the Kaggle and HackerRank competitions for fun stuff and the opportunities to compete and learn.
Learning never truly ends. Here are some pointers to help you with continuous learning
- Read some excellent blogs. Both Christopher Olah's blog & Andrew Karpathy's Blog do a great job of explaining basic concepts and recent breakthroughs
- Follow influencers on Twitter. Here are a few to get started @drfeifei, @ylecun, @karpathy, @AndrewYNg, @Kdnuggets, @OpenAI, @googleresearch. (see: Who to follow on Twitter for machine learning information ? )
- Joining the Google+ Deep Learning Community, by Yann Lecunn, is a good way to keeping in touch with innovations in deep learning as well as communicating with other deep learning professionals and enthusiasts.
See ChristosChristofidis/awesome-deep-learning, a curated list of awesome Deep Learning tutorials, projects and communities for more fun.