promptdojo_

The four breakage classes — sort any LLM failure before you touch the prompt — step 8 of 9

Write classify_failure(trace) that returns one of "retrieval", "prompt", "hallucination", or "parse".

Apply the priority order from the chapter:

  1. If trace["retrieved_chunks"] is non-empty AND trace["retrieved_chunks_match_query"] is False"retrieval".
  2. Else if trace["raw_output"] != trace["output_after_postprocess"]"parse".
  3. Else if trace["retrieved_chunks"] is empty (the list []) → "hallucination".
  4. Else → "prompt".

Three traces run for you, one per branch. Expected output:

retrieval
parse
hallucination

full-screen editor opens — close anytime to keep reading.