Machine learning

What is machine learning? Tom Mitchell‘s definition is

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

Machine learning is about asking how we can program computers without providing specific algorithms.

Domains

• Supervised learning
• Unsupervised learning
  • Self supervised learning
• Reinforcement learning

Foundations

• Information theory
• Statistics (Machine learning vs)
• Physics based interpretation of machine learning

Domains of application

• Machine learning for healthcare
• Machine learning for physical sciences
• Machine learning for scientific discoveries
• Machine learning for social scientists

Approaches

• Automated machine learning
• Cluster analysis
• Dimensionality reduction
• Image completion
• One shot learning
• Recommender system
• Training dataset
• Interpretation of machine learning models

Methods

• Artificial neural network
• Boosting (machine learning)
• Decision tree
  • Random forest
• Deep learning
• EM algorithm
• Naive Bayes classifier
• Cross validation and Stability selection

Books

• The elements of statistical learning
• An Introduction to Statistical Learning with applications in R - Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani
• free machine learning books
• Machine Learning by Tom M
• Bayesian Reasoning and Machine Learning
• https://probml.github.io/pml-book/ - Probabilistic Machine Learning series by Kevin Murphy
  • Machine Learning: A Probabilistic Perspective
  • Probabilistic Machine Learning: An Introduction
  • Probabilistic Machine Learning: Advanced Topics
• a reading list by Michael I
• The hundred-page machine learning book

Courses

• Machine Learning University - a curated list of course resources.
• Andrew Ng’s artificial intelligence | machine learning - open lecture.
  • http://ml-class.org/ - open course
  • http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a6d-4052-937d-cb017338d7b1
• Tom Mitchell
• Practical Machine Learning by Michael Jordan (computer scientist)
• Machine Learning Summer School 2011 - Bordeaux
• Machine Learning Fall 2009 by Carlos Guestrin
• http://www.youtube.com/playlist?list=PLD0F06AA0D2E8FFBA - Machine Learning by mathematical monk
• CALTECH: learning from data
• Machine Learning and Probabilistic Graphical Models Course by Sargur Srihari
• Kaggle Python Tutorial on Machine Learning

Tools and libraries

• Python
• scikit learn
• http://mldemos.epfl.ch/ - MLDemos
• http://harthur.github.com/brain/ - Javascript supervised machine learning library
• GraphLab
• wiki - vowpal wabbit, a fast online learning algorithm
• http://www.cs.waikato.ac.nz/ml/weka/index.html - Weka
• http://blog.mashape.com/post/48074869493/list-of-machine-learning-apis

Tutorials and Articles

• How to Write a Spelling Corrector by Peter Norvig
• The Ideal Large Scale Learning Class
• Self-Study Guide to Machine Learning
• Machine learning in 10 pictures
• kaggle:Learning from the best
• Machine Learning Done Wrong
• Approaching (Almost) Any Machine Learning Problem | Abhishek Thakur
• Machine Learning is Fun!
  • Machine Learning is Fun! Part 2
  • Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks
  • Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning
• What I learned from Deep Learning Summer School 2016
• Model evaluation, model selection, and algorithm selection in machine learning - Part I
  • Model evaluation, model selection, and algorithm selection in machine learning - Part II
  • Model evaluation, model selection, and algorithm selection in machine learning - Part III - Cross-validation and hyperparameter tuning
• Google developers: Machine Learning Recipes
• Rules of Machine Learning: Best Practices for ML Engineering

Tips

• Lessons learned debugging ML models
• https://twitter.com/marktenenholtz/status/1501905740813848582
  • Tabular: XGBoost/LightGBM/RF
  • Time series: XGBoost/LightGBM/RF
  • Image: ResNet/EffNet
  • Text: RoBERTa
  • Audio: ResNet/EffNet
• Avoid these pitfalls when interpreting your machine learning model.👇

Websites and blogs

• http://metaoptimize.com/qa/questions/3163/good-machine-learning-blogs

Talks and lectures

• http://blog.videolectures.net/100-most-popular-machine-learning-talks-at-videolectures-net/
• mathematical monk: Machine learning
• Advice for applying Machine Learning by Andrew Ng

Other resources

• Machine Learning Cheat Sheet (for scikit learn)
• Machine learning classifier gallery

References

Reviews

• Machine learning: Trends, perspectives, and prospects

Explaining the prediction

• “Why Should I Trust You?”: Explaining the Predictions of Any Classifier

To read

• https://engraved.ghost.io/why-machine-learning-algorithms-are-hard-to-tune/