The Lectures · Course I
Machine Learning
One classic algorithm per lecture, and each one gets three treatments: the mathematics, derived rather than asserted, with every update rule earned from a loss function and the chain rule; the implementation, in plain Python, structured so that each line of code answers to a line of the derivation; and an interactive plate to descend, train, or break.
The instrument
The code is pyml, whose GitHub
description reads, in full, ML algs implemented in python for fun; I love
reinventing the wheel. That was meant as a joke and has become a mission
statement: frameworks are how you ship, from scratch is how you understand. When
model.fit(X, y) works first time you have learned that the API is well
designed; when your own gradient descent diverges, oscillates, then walks down the
loss surface after you fix a transposed matrix, you have learned what fitting is.
The ground rules follow from that. No ML frameworks anywhere. Linear algebra is
fair game, automatic differentiation is not, because deriving the gradients is the
entire point. Every analytic gradient is checked entry by entry against finite
differences in the test suite, since a derivation that cannot survive a central
difference is not a derivation but a hope. And an algorithm ships only
when its lecture does.
That last rule is structural rather than principled. From-scratch ML repositories are the most-started and least-finished projects on GitHub: everyone writes linear regression, far fewer write the second algorithm, and almost nobody writes tests, because the fun is front-loaded and the discipline is not. Chaining each implementation to a published lecture means an unfinished algorithm is a visibly unfinished page on this site. The honest limitation is the genre's own: raw Python is orders of magnitude slower than anything you would deploy, and the numerical-stability machinery (careful initialisation, log-sum-exp, conditioning) is deliberately skipped unless it becomes a lesson in its own right. Every omission is a thing you can now see a framework doing for you, which is the point, but this is a teaching instrument and not a library.
The arc
-
I.
Linear Regression & Gradient Descent published
Least squares derived from the two partials, the normal equations, and the one-line update that survives every later model.
Plate The loss surface itself: contours, the closed-form optimum, and four optimizers walking down to it at a learning rate you choose.
-
II.
A neural network from scratch published
Backpropagation derived honestly as the chain rule run backwards, then implemented raw, with every gradient checked against finite differences.
Plate A two-layer network learning a boundary no line can draw, with the backward pass stepped through layer by layer.
-
III.
Decision trees, and a model with no gradient published
Greedy search over thresholds instead of descent, impurity as the thing being reduced, and the axis-aligned bias that arrives free with the choice of question.
Plate A tree grown one split at a time beside the plane it is carving, the two linked because every node of a tree is a rectangle; both impurity criteria scored at once on the split about to be made.
-
IV.
Clustering without an answer key published
k-means and a Gaussian mixture as the same alternation with different commitments, and a dataset on which the correct answer scores worse than the wrong one.
Plate The same points clustered twice, k-means against EM, stepped one phase at a time, with responsibilities drawn as shading and the seed exposed so local optima can be found on purpose.