GitHub Actions for model checks — what a green check does and doesn't mean (step 5/7) · git and github cli

promptdojo_

Claude wrote scripts/eval_gate.py for your CI and it "worked" for two weeks — every run green. Then you read it: the verdict on line 5 only checks pytest. A 0.71 model sailed past a 0.80 threshold fourteen times. Fix line 5 so the run is green only when tests pass AND eval_score clears THRESHOLD.

Expected output:

red X

The break is on line 5 — but read the whole snippet first.

Expected output:

red X

The break is on line 5 — but read the whole snippet first.

full-screen editor opens — close anytime to keep reading.

GitHub Actions for model checks — what a green check does and doesn't mean — step 5 of 7