Machine Learning Reference List

Here's a curated list of references/resources that you can use to teach yourself Machine Learning, annotated with prerequisites and recommendations to help you choose. It's a work in progress and I'll be updating it every so often, so be sure to keep checking!

Courses

A list of highly recommended short courses and lecture video series.

Mathematics for Machine Learning

Machine Learning requires some mathematical prerequisites for you to understand what's going on. Generally, experience with statistics, probability, linear algebra, and multivariate calculus are hard requirements. Reading papers and books is hard without the prerequisites, so we cannot stress how important it is to go through them first! If you're pressed with time, I recommend running through just CMU 10-606. Otherwise, the other things listed would provide really good resources.

Course	Source	Notes	Prerequisites
[10-606] Mathematical Background for Machine Learning	Carnegie Mellon University	Teaches the mathematical foundations. Take this if unsure where to go. No needed prior knowledge, but previous classes on statistics, linear algebra, and calculus would help.	None.
Essence of Linear Algebra	3Blue1Brown	My favorite linear algebra short course online, with an emphasis on geometric interpretations and intuitions. Recommended binge watch.	None.
Computational Linear Algebra	fast.ai	A more practical linear algebra class with emphasis on practical intuition and coding. Teaches you how to code the algorithms for those who like to see numbers in action.	None.
Linear Algebra Review	Carnegie Mellon University	A quick review on the guts of linear algebra. Only take it if you've taken a previous course.	Previous linear algebra course.
[10-705] Intermediate Statistics	Carnegie Mellon University	Works nicely as a theoretical take on statistical theory. Coers topics such as bayesian inference and nonparametric testing. This is not a basic statistics course, and is intended for a deep dive in intermediate statistics! Recommended for intermediate students.	Previous statistics and probability course.

Introduction to Machine Learning

Basic Machine Learning. Assumes the required mathematical background is met, ideally CMU 10-606 has been taken as a precursor. Go take CMU 10-701 if unsure where to go.

Course	Source	Notes	Prerequisites
[10-701] Introduction to Machine Learning	Carnegie Mellon University	My current go-to recommendation for an introduction to machine learning. Solid offering.	Statistics, linear algebra, and calculus (multivariate). CMU 10-606 would work as a precursor.
[CS229] Introduction to Machine Learning	Stanford University	Used to be my go-to, but is outdated (the video quality says it all). Go to CMU 10-701 unless you like seeing Andrew Ng prove theorems on six blackboards at once.	Statistics, linear algebra, and calculus (multivariate). CMU 10-606 would work as a precursor.
[CS155] Introduction to Machine Learning	California Institute of Technology	Caltech's intro to machine learning, which Stan recommends. Yisong Yue teaches really well.	Statistics, linear algebra, and calculus (multivariate). CMU 10-606 would work as a precursor.

Deep Learning

Neural networks and everything under it's umbrella. Requires good foundations! Some background in basic machine learning would help. Go take Hugo Larochelle's course for a more theoretical (proof-based) introduction.

Course	Source	Notes	Prerequisites
Neural Networks	Universitè de Sherbrooke	Hugo Larochelle's neural networks class. My go-to recommendation for people who want a good theory-based introduction with complete proofs and intuitions.	Good linear algebra and multivariate calculus, plus statistics and information theory.
Neural Networks	Coursera	The coursera neural networks course from the godfather Geoffrey Hinton himself!	Good linear algebra and multivariate calculus, plus statistics and information theory.
Neural Networks	3Blue1Brown	Not an in-depth deep learning course but rather a series of videos that help you visualize and understand the intuitions and mechanics behind neural networks. Recommended binge-watch! Can be used as a primer to get your feet wet.	Just calculus is fine.

Natural Language Processing

Courses on Natural Language Processing (with an emphasis on deep learning methods). I learned from the old CS224N course, but we've added the new one here.

Course	Source	Notes	Prerequisites
[CS224N] Introduction to Natural Language Processing with Deep Learning	Stanford University	The Winter 2019 version of the Deep Learning + NLP course offered by Christopher Manning and Abigail See. Highly recommended.	A previous machine learning course. A previous deep learning course is appreciated.
[CS224N] Introduction to Natural Language Processing with Deep Learning	Stanford University	The older, time-tested version of the CS224N course, added here in case you need it.	A previous machine learning course. A previous deep learning course is appreciated.

Computer Vision

Courses on Computer Vision (with an emphasis on deep learning methods).

Course	Source	Notes	Prerequisites
[CS231N] Convolutional Neural Networks for Visual Recognition	Stanford University	Stanford's computer vision class with an emphasis on deep learning techniques. Taught by Fei-Fei Li, Justin Johnson, and Serena Yeung.	A previous machine learning course. A previous deep learning course is appreciated.

Data Science

We're actually making some data science lecture videos so watch this space as it's coming soon!

Books

Some of my highly recommended books.

Machine Learning and Deep Learning

• Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016) - Written by some of the greatest names in AI Research today, this book is the best I can recommend for an introduction to Deep Learning.
• Pattern Recognition and Machine Learning by Christopher Bishop (2006) - The OG Machine Learning Book. A tad bit outdated, but it does work really great as a classical machine learning textbook.

Data Science

• Python Data Science Handbook The book introduces the core libraries essential for working with data in Python: particularly IPython, NumPy, Pandas, Matplotlib, Scikit-Learn

Conference and Journal Papers

We're making a separate list of good papers to read. Coming soon!

Blogs and Podcasts

A collection of blogs, blog posts, and podcasts that talk ML and Deep Learning.

Websites and Trackers

• Papers With Code - A tracker with SOTA leaderboards for multiple tasks in different areas of research.
• Tracking The Progress of NLP - A tracker that's regular updated with the state-of-the-art in various NLP tasks, datasets, techniques, etc.

Blog Posts

• The Matrix Calculus You Need For Deep Learning - A "paper" attempting to explain all the matrix calculus you need to understand deep learning. The learning curve might be a tad bit too steep so I suggest it as a side-material when taking a math course.
• The Best of Universal Sentence/Word Embeddings - From the blog of Thomas Wolf, an NLP-Deep Learning researcher. Reviews the current state-of-the-art techniques in word embedding and sentence embedding techniques.