Additional Text Resources (will reference some content)
Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville.
On foundations of modern neural networks. Available free online.
- Useful python tutorials
- Python Editors and IDEs
- Notes on setting up environments
- Three python libraries you will need:
Python library for mathematical computing.
Builds on NumPy with additional scientific computing facilities.
Python library for plotting/graphing, intentionally close to Matlab's plotting facility.
All written assignments will be released in LaTeX, and it is highly recommend that you use LaTeX. Yes, it's another programming language, but the assignments will provide most of the required formatting. It is truly the best way to format math. There are MANY resources available online (just google!) -- here are two links to get you started:
3Blue1Brown Essence of Calculus. Gorgeous series of 11 videos that introduce the intuition behind the development of calculus. Great overview/review.
(A little more tangential (pun intended), but here's a followup vid with some additional nice intuition about derivatives.)
We will take derivatives in this class. Do not be put off if you need some review. Here is the general link for Khan Academy Differential Calculus, but the following two links are most relevant for reviewing the basic mechanics of differentiation we'll use in this course:
We will see integrals (particularly with expectations of continuous random variables), although we will only do a little actual integrating. It is important to understand the intuitions underlying integration. The above "3Blue1Brown Essence of Calculus" will give a lot of this.
- Here is the top-level link to the Khan Academy Integral Caculus Series. You can browse for particular topics to review
- I do recommend the first three videos of definite integral evaluation to review how to evaluate basic definite integrals (along with the second fundamental theorem of calculus).
3Blue1Brown Essence of Linear Algebra. Again, really well-done series, 15 videos that develop great intuitions of linear algebra, emphasizing the underlying visualizable geometry (when dimensions are low enough!).
Gilbert Strang MIT OpenCourseWare Linear Algebra Course. The full course, 36 videos. Excellent and very accessible, good for first time exposure, and for reviewing concepts.
Linear Algebra Common Lounge quick overview. This summarizes as a very quick reference some common linear algebra topics. (Warning: under the Vector Spaces topic, when they say "lengh 2 vector" they mean 2-dimensional, NOT the magnitude of the vector; I think this is unfortunate wording.)
Multivariable (vector) Calculus
- I don't require vector calculus as a pre-requisite for this class, but we will be working with multivariable functions, taking derivatives of these functions, and representing the gradient, so we will need a little bit of vector calc (there's much more to vector calc than what we will use). If this is new to you, don't worry! The following Khan Academy videos provide what we need of vector calculus for this course, and I will also review along the way during the course.
- If just thinking about multivariable functions is new to you, start with this short video: Introduction to multivariable calculus.
- Then take a look at this series of Khan Academy videos on: Derivatives of Multivariable Functions. We only need the following two components from this series:
- Partial derivatives: watch the four videos of section 1 (Partial derivatives). And check out these two Khan Academy "articles":
Intro to partial derivatives and
Second partial derivatives
- Gradient: The gradient is just a vector representation of a set of partial derivatives. If this is new (or you need review), watch the first three videos of section 2 (Gradient and directional derivatives). And check out this Khan Academy "article": The gradient
- We will not be using directional derivatives or any of the other derivative-related operators (grad, div, curl, Laplace, etc...). (although they're all cool in their own right!)
Triangle of Power! More for fun, this is another nice 3Blue1Brown video about the relationship between base, exponentiation and logarithm, and why notation matters!
Misc general introductory machine learning
General ML-related blogs
Probability and Statistics
Probabilistic Models of Cognition
Very nice collection of discussions on various probability topics by Noah D. Goodman and Joshua B. Tenenbaum, emphasizing a Bayesian perspective.
math.stackexchange: why use Bayes' Theorem...
A nice response to a question about the difference between being Frequentist versus Bayesian and the role of Bayes' Theorem; also the difference between descriptive versus inferential statistics.
Lessons in proper assessment of machine learning
Interpretable Machine Learning
Support Vector Machines (SVMs)
Sampling for probabilistic inference
Mostly for fun...
Demonstrations of other interesting approaches to machine learning not covered in this course
Fun and informative web-based implementation of a genetic algorithm that evolves wheeled vehicles.