Additional Text Resources (may use some content)
Online book by Ian Goodfellow, Yoshua Bengio and Aaron Courville on foundations of modern neural networks.
- Useful python tutorials
- Python Editors and IDEs
- Notes on setting up environments
- Three python libraries you will need:
Python library for mathematical computing.
Builds on NumPy with additional scientific computing facilities.
Python library for plotting/graphing, intentionally close to Matlab's plotting facility.
All written assignments will be released in LaTeX, and it is highly recommend that you use LaTeX. Yes, it's another programming language, but the assignments will provide most of the required formatting. It is truly the best way to format math. There are MANY resources available online (just google!) -- here are two links to get you started:
3Blue1Brown Essence of Calculus. Gorgeous series of 11 videos that introduce the intuition behind the development of calculus. Great overview/review.
We will take derivatives in this class. Do not be put off if you need some review. Here is the general link for Khan Academy Differential Calculus, but the following two links are most relevant for reviewing the basic mechanics of differentiation we'll use in this course:
We will see integrals (particularly with expectations of continuous random variables), although we will only do a little actual integrating. It is important to understand the intuitions underlying integration. The above "3Blue1Brown Essence of Calculus" will give a lot of this.
- Here is the top-level link to the Khan Academy Integral Caculus Series. You can browse for particular topics to review
- I do recommend the first three videos of definite integral evaluation to review how to evaluate basic definite integrals (along with the second fundamental theorem of calculus).
Multivariable (vector) Calculus
- I don't require vector calculus as a pre-requisite for this class, but we will be working with multivariable functions, taking derivatives of these functions, and representing the gradient, so we will need a little bit of vector calc (there's much more to vector calc than what we will use). If this is new to you, don't worry! The following Khan Academy videos provide what we need of vector calculus for this course, and I will also review along the way during the course.
- If just thinking about multivariable functions is new to you, start with this short video: Introduction to multivariable calculus.
- Then take a look at this series of Khan Academy videos on: Derivatives of Multivariable Functions. We only need the following two components from this series:
- Partial derivatives: watch the four videos of section 1 (Partial derivatives). And check out these two Khan Academy "articles":
Intro to partial derivatives and
Second partial derivatives
- Gradient: The gradient is just a vector representation of a set of partial derivatives. If this is new (or you need review), watch the first three videos of section 2 (Gradient and directional derivatives). And check out this Khan Academy "article": The gradient
- We will not be using directional derivatives or any of the other derivative-related operators (grad, div, curl, Laplace, etc...). (although they're all cool in their own right!)
Triangle of Power! More for fun, this is another nice 3Blue1Brown video about the relationship between base, exponentiation and logarithm, and why notation matters!
General ML-related Blogs
Probability and Statistics
Probabilistic Models of Cognition
Very nice collection of discussions on various probability topics by Noah D. Goodman and Joshua B. Tenenbaum, emphasizing a Bayesian perspective.
math.stackexchange: why use Bayes' Theorem...
A nice response to a question about the difference between being Frequentist versus Bayesian and the role of Bayes' Theorem; also the difference between descriptive versus inferential statistics.
Support Vector Machines (SVMs)
Sampling for Probabilistic Inference
Misc general introductory machine learning
Lessons in Proper Assessment of Machine Learning
Mostly for fun...
Demonstrations of other interesting approaches to machine learning not covered in this course
Fun and informative web-based implementation of a genetic algorithm that evolves wheeled vehicles.