Baselines before fancy models (step 1/7) · supervised learning workflows

Baselines before fancy models

Chapter 21 taught evals-before-prompts. The classical-ML version of the same discipline: baselines before models. A baseline is a predictor so simple it can't be wrong about why it works — and it sets the bar every real model must clear.

The baseline ladder

Run the editor. Two rungs:

Majority class / predict-the-mean. Zero intelligence. On imbalanced data its accuracy is embarrassingly high (chapter 38), which is exactly why it must be printed next to every model score.
One rule from domain knowledge. "Three or more support tickets → churn risk." Ten minutes with someone who knows the business, one if statement. This rung is the humbling one — it's often surprisingly close to the fancy model.

Then, and only then, the simplest trained model — in sklearn, LogisticRegression for classification, LinearRegression for regression:

from sklearn.linear_model import LogisticRegression
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
model.score(X_test, y_test)         # accuracy vs the baselines

fit then predict/score — that API shape is uniform across sklearn, which is why the baseline ladder is cheap: swapping LogisticRegression for RandomForestClassifier is one line.

Why the ladder wins arguments

Every rung answers a different question. Majority class: "is the metric fooling us?" One rule: "does pattern exist beyond common sense?" Linear model: "do the features carry signal at all?" When the gradient-boosted forest finally arrives, its score means something — it's the delta over each rung, not a lonely number. And when the fancy model barely beats the one-rule baseline, you just saved a quarter of engineering: ship the rule (it's explainable, fast, and free).

Where AI specifically gets this wrong

Straight to the ensemble. Ask Cursor for a churn model and you'll get XGBoost-shaped code with no baseline in sight. Demand the ladder in the same script, one table of scores.
Baselines evaluated on train. The bar only counts on the held-out set — same split for every rung, or the comparison is garbage.
Skipping the domain rule. The one-rule baseline needs a human who knows the business — that's you, not the model.
Missing what the rule detects. The domain rule is also your best leakage detector: if one trivial rule scores 99%, chapter 38 says hunt the leak.