promptdojo_

FastAPI inference shape — step 5 of 7

Three requests, three model loads — the counter proves this endpoint deserializes the model on EVERY request (imagine 800ms each; now imagine 50 requests a second). Fix the shape: load once at startup into module state, and let the endpoint just predict. The counter must read 1.

The break is on line 8 — but read the whole snippet first.

full-screen editor opens — close anytime to keep reading.