Draft–Verify
System 1 drafts quickly; System 2 checks or corrects the draft.
Problem
Having a reasoning model write every output from scratch is slow and expensive, but shipping a fast model’s drafts unchecked lets its confident mistakes through. Most drafts are fine; the cost is in finding the few that aren’t.
- Checking a draft is usually faster and cheaper than writing it.
- Fast models get most outputs right but fail unpredictably on some.
- A verifier that rewrites everything erases the savings.
- Some errors are only visible against source data the drafter didn’t use well.
Solution
Let a fast model produce every draft. Pass the draft, the original request and the source material to a reasoning model with a narrow job: verify specific claims and return a verdict per claim (supported, wrong or missing) plus minimal corrections. Accept the draft as-is when everything checks out, apply the small corrections when they don’t, and regenerate with the strong model only when the draft is beyond repair.
Diagram
The verifier returns a verdict and edits, not a new answer. Most traffic takes the top path with no changes.
When to use
Use it when
- High-volume outputs that are mostly correct already
- Claims can be checked against provided sources or data
- Verifying is clearly cheaper than generating, such as long drafts with a few facts
- Errors are costly but latency can absorb one check
Avoid it when
- The fast model is wrong so often that most drafts get regenerated
- There is no source of truth to check against
- Real-time replies where even one check is too slow; see Respond-then-Refine
Trade-offs
Gains
- Strong-model accuracy at close to fast-model cost
- Per-claim verdicts show exactly what was wrong
- Correction rates tell you when the fast model has drifted
Costs
- The verifier can only check what's checkable
- Two models to version and evaluate together
- Subtle errors of omission are hard to verify
Worked example
A finance-ops agent writes monthly variance commentary for 40 cost centers from the ledger data.
- Fast modelDrafts 40 short commentaries in about 12 seconds total.
- VerifierChecks each number and claim against the ledger. 34 drafts pass with no changes.
- Verifier5 drafts get small fixes. One example: “travel up 12%” is corrected to 21%, the actual change from $48k to $58k.
- Verifier1 draft blames a variance on headcount, but the ledger shows a one-time vendor credit.
verdict: regenerate. The strong model rewrites that one. - Weekly metricsCorrection rate is 15%. A month later it rises to 30% after a prompt change, which flags a regression before anyone reads a wrong report.
Failure modes
| Failure | Symptom | Mitigation |
|---|---|---|
| Verifier rewrites everything | Output looks like the strong model wrote it; no savings | Ask for verdicts and minimal edits only; measure edit size |
| Unverifiable claims slip through | Opinions or causes pass because there's no data to contradict them | Flag claims with no supporting source as "unsupported" |
| Verifier trusts the draft's framing | Checks numbers but misses a wrong conclusion | Include "does the conclusion follow from the data?" as a check item |
| Shared source errors | Both models agree on wrong source data | Validate source data upstream; verification can't fix bad inputs |