Deep learning

Machine learning methods that use deep Neural networks, which are trained via Stochastic gradient descent in most cases.

Sometimes it can be an unnecessary complexity¹².

Why deep learning works?

What are the lessons? Lessons from deep learning

Can complex systems or network science perspectives help? Complex systems and deep learning.

Topics

• Architectures and methods
  • Autoencoder
  • Convolutional neural network
  • Deep generative models
  • Differentiable neural computers
  • Generative adversarial network
  • Geometric deep learning
    • Deep learning and graphs
  • Recurrent neural network
  • Residual Learning
  • word2vec | Continuous embedding | Word embedding
• Brain and deep learning
  • Brain and deep learning
• Theory of deep learning
  • Why deep learning works

Courses

• CS231n: Convolutional Neural Networks for Visual Recognition
  • Videos
• CS224n: Natural Language Processing with Deep Learning
  • Videos
• Coursera: deep learning specialization
• Microsoft Professional Program for Artificial Intelligence track
• Google: deep learning
• Hugo Larochelle: Neural networks
• Geoffrey Hinton Coursera videos
• Probabilistic Graphical Model by Daphne Koller
• Deep learning by Lex Fridman: https://github.com/lexfridman/mit-deep-learning

Books

• http://www.deeplearningbook.org/
• The Principles of Deep Learning Theory

Tools

• Evaluation of Deep Learning Toolkits
• TensorFlow
• Keras

Visualization

• https://github.com/rewonc/pastalog

Talks and tutorials

• A Tutorial on Deep Learning Part 1: Nonlinear Classifiers and The Backpropagation Algorithm by Quoc Le
  • Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks
• Deep Learning Basics: Neural Networks, Backpropagation and Stochastic Gradient Descent
• ufldl stanford:Deep Learning Tutorials
• sjchoi86:Deep learning tutorials
• Unsupervised Feature Learning and Deep Learning by Andrew Ng
• Use Google’s Word2Vec for movie reviews
• Using convolutional neural nets to detect facial keypoints tutorial
• Deep Learning: Doubly Easy and Doubly Powerful with GraphLab Create - GraphLab
• ICML2013 Deep learning tutorial by Yann LeCun and Marc’Aurelio Ranzato

Articles

• http://fastml.com/deep-learning-made-easy/
• So You Wanna Try Deep Learning?
• Recommending music on Spotify with deep learning
• Inceptionism: Going Deeper into Neural Networks

References

• https://github.com/terryum/awesome-deep-learning-papers

Review

• Deep learning by Yann LeCun, Yoshua Bengio, and Geoffrey Hinton
  • Critique of Paper by “Deep Learning Conspiracy” (Nature 521 p 436 by Jürgen Schmidhuber

Augmenting small datasets with transformations

• Some Improvements on Deep Convolutional Neural Network Based Image Classification

Noise

• ADDING GRADIENT NOISE IMPROVES LEARNING FOR VERY DEEP NETWORKS - Thermostat

Adversarial examples

• Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
  • Youtube
• Universal adversarial perturbations

About baselines

• Simple Baseline for Visual Question Answering

Natural language processing

• ATTENTION AND MEMORY IN DEEP LEARNING AND NLP

Binarization

• Deep neural networks are robust to weight binarization and other non-linear distortions

Others

• http://blog.kaggle.com/2012/11/01/deep-learning-how-i-did-it-merck-1st-place-interview/
• Building High-level Features Using Large Scale Unsupervised Learning
• Deep Neural Networks for Acoustic Modeling in Speech Recognition
• ImageNet Classification with Deep Convolutional Neural Networks
• Large Scale Distributed Deep Networks
• How transferable are features in deep neural networks?
• Human-level control through deep reinforcement learning
  • AI masters 49 Atari 2600 games without instructions