Embeddings as features: the bridge between the two courses
Chapter 22 used embeddings for retrieval — find the chunk nearest the query. This lesson reframes the same vectors for the ML side: an embedding is a learned feature vector, and everything this arc taught about features applies to it.
One representation, four jobs
Run the editor: toy embeddings plus cosine similarity give you a zero-training classifier (nearest anchor wins). The same vectors power:
- Retrieval — nearest neighbors to a query (chapter 22).
- Classification — nearest labeled anchor, exactly as run here; or feed the embedding into a supervised model as its feature row (chapter 39's X can be embeddings).
- Clustering — k-means over embeddings groups texts by meaning; the "read the clusters" step becomes "read ten texts per cluster."
- Deduplication / anomaly — near-identical vectors are dupes; a vector far from everything is an outlier worth reading.
This is why embeddings earn the "features for unstructured data" title: they turn text (or images, or audio) into the fixed-length numeric rows every tool in this arc expects.
The rules that carry over
- Same model for everything you compare — chapter 22's law. Vectors from different embedding models live in unrelated spaces; mixing them makes cosine garbage.
- Version your embedding model like a schema. Re-embedding a corpus with a new model invalidates every stored vector — that's a migration (chapter 37's contract discipline), not a hot swap.
- Leakage still applies. Embedding text that contains the label ("customer churned, closing ticket") and training on it is chapter 38's cheat-feature, at 1536 dimensions where it's harder to spot. Audit what text goes in.
Where AI specifically gets this wrong
- Embedding the label along with the input. Generated ticket-classifier code cheerfully embeds the whole record — including the resolution field the model exists to predict.
- Mixing embedding versions in one index. Half the corpus on the old model, half on the new: retrieval quietly degrades and nobody can say why (chapter 22's junior-engineer bug at dataset scale).
And the one your invoice catches before you do: every embedded row is an API call (or a local model pass) — chapter 22's budget math applies to training pipelines too.