skip to content
promptdojo
_
[ save your spot ]
[ follow on x ]
Pagination, retries, and checkpoints — step 2 of 7
ch 37 · dataset formats, ingestion, and validation pipelines
2/7
promptdojo
_
›
phase 08 · ai/ml engineering
›
ch 37 · dataset formats, ingestion, and validation pipelines
lesson 2 of 5 · pagination, retries, and checkpoints
step 2 of 7 in this lesson
When should a pipeline persist the cursor for a page?
1
Before fetching the page, so the run looks faster.
2
After the page rows are validated and written.
3
Only after the entire pipeline finishes.
4
Never; retries should always start from page one.
check
ch 37 · dataset formats, ingestion, and validation pipelines
2/7
promptdojo
_
›
phase 08 · ai/ml engineering
›
ch 37 · dataset formats, ingestion, and validation pipelines
lesson 2 of 5 · pagination, retries, and checkpoints
step 2 of 7 in this lesson
When should a pipeline persist the cursor for a page?
1
Before fetching the page, so the run looks faster.
2
After the page rows are validated and written.
3
Only after the entire pipeline finishes.
4
Never; retries should always start from page one.
check
park a thought