FastAPI inference shape — step 1 of 7

The FastAPI inference shape

A trained model becomes a product the moment it answers HTTP. The standard Python shape for that is FastAPI — and it's a reunion tour of this course: chapter 12's HTTP, chapter 14's pydantic validation, chapter 45's registry, one endpoint.

The real code

from fastapi import FastAPI
from pydantic import BaseModel
# these two are YOUR modules — the ch.45 registry loader and the
# exact feature transform training used (never a reimplementation):
from myproject.registry import load_model
from myproject.features import features

app = FastAPI()
model = load_model("churn", stage="production")   # ONCE, at startup

class PredictRequest(BaseModel):
    tickets: float
    logins: float

class PredictResponse(BaseModel):
    score: float
    model_version: int

@app.post("/predict", response_model=PredictResponse)
def predict(req: PredictRequest):
    x = features(req)                  # same transform as training (ch.45)
    return PredictResponse(score=model.predict_proba(x),
                           model_version=model.version)

Run the editor for the same contract in miniature: validate → predict → respond, with the model "loaded" once as module state, not per request.

Load-bearing decisions

1. Load once, at startup. Deserializing a model per request turns milliseconds into seconds and melts under load. The generated-code version of this bug is pickle.load inside the endpoint — chapter 39's train-once/predict-cheap asymmetry, violated in production.
2. Pydantic on both doors. The request model rejects malformed input with a structured 422 before it reaches the model (chapter 45's contract at the serving door); the response model guarantees the caller a stable shape.
3. Version in every response. When chapter 47's monitoring finds a bad prediction, model_version in the response (and in the logs) is the difference between an investigation and a shrug.

One more habit that pays immediately: a /health endpoint that confirms the model actually loaded — it's what deploy tooling and chapter 47's alerts will probe.

Where AI specifically gets this wrong

• Model loading inside the endpoint. The classic. Move it to startup; the diff is three lines and 100× latency.
• Dict-in, dict-out endpoints. Skipping pydantic "for simplicity" re-opens every ch.12 KeyError as a production 500 — with a customer on the other end.
• Re-implemented feature transforms. Chapter 45.2's skew, delivered by HTTP. Import the training transform; never translate it.