Vectors, matrices, dot products — you can read this
ML math has a terrifying reputation and a small working core. For reading real code, you need three objects and one operation, and you already met two of them in chapter 22.
The three objects
- Vector — a list of numbers. One user's features. One
embedding. Shape
(n,). - Matrix — a grid of numbers. A whole dataset (rows = samples,
columns = features). A layer's weights. Shape
(rows, cols). - Tensor — the generic word for "n-dimensional grid." A vector is a 1-D tensor, a matrix 2-D, a batch of images 4-D. Chapter 42 makes these first-class.
The one operation: the dot product
Run the editor. dot(features, weights) multiplies pairwise and
sums — a weighted sum. That's it. And that's the atom of nearly
everything:
- A linear model's prediction is one dot product per row.
- A neural network layer is many dot products in parallel (that's all matrix multiplication is: every row of A dotted with every column of B).
- Chapter 22's cosine similarity is a dot product of normalized vectors — you already ran one.
- Attention (chapter 44) scores tokens against each other with — dot products.
Negative weight = pushes the score down; zero = ignored; big magnitude = the model cares a lot. When you can read a weighted sum, you can read what a linear model believes.
Matrix shapes: the compatibility rule
Matrix multiply A @ B requires inner dimensions to agree:
(n, k) @ (k, m) -> (n, m). Dataset (1000, 12) times weights
(12, 1) gives 1000 predictions. Nearly every deep-learning crash
message is this rule being violated — the fix is chapter 35's
habit: print shapes.
Where AI specifically gets this wrong
- Mystifying the math in comments. Cursor writes "compute
logits via affine transformation" over what is
X @ w + b— a batch of weighted sums plus an offset. Translate ruthlessly. - Shape errors patched with
.reshape. Reshaping until the crash goes away, without knowing which dimension means what, is how silently-transposed data ships. Name your dims in comments:# (batch, features).
A lesser sin, still worth catching: dot products reimplemented as loops. Correct but slow and unreadable at scale — the vectorization lesson from chapter 35 applies doubly here.