Respond-then-Refine
An instant first answer from System 1, followed by a refined System 2 version.
Problem
Users want an answer now, but the best answer takes a reasoning model many seconds or minutes. Making them wait loses their attention; giving only the fast answer leaves quality on the table for the questions where depth matters.
- Perceived speed matters as much as actual speed.
- A quick answer is often good enough, and the user moves on.
- Changing an answer after the user has read it can confuse or erode trust.
- Running the slow model on every request is wasteful if users rarely need it.
Solution
Return a fast model’s answer immediately, clearly labeled as a first take. Start the reasoning model in the background on the same request, with the fast answer as context. When it finishes, show the refined answer next to or in place of the first one, and highlight what changed and why. If the refinement agrees with the first answer, say so briefly instead of repeating it. Trigger the background step only when the request qualifies, such as when the user stays on the page or the question is marked high value.
Diagram
The user is never blocked. The refined answer arrives later and explains what changed, so the update builds trust instead of eroding it.
When to use
Use it when
- Interactive tools where waiting 20+ seconds loses the user
- Analysis questions where a first take is useful and a deeper one is better
- Users can see an answer update in place (dashboards, chat with streaming UI)
- You can tell which requests deserve the background pass
Avoid it when
- The first answer will trigger an irreversible action
- Channels that can't update a sent answer, such as SMS or email
- Errors in the fast answer could mislead before the correction arrives
Trade-offs
Gains
- Feels instant without giving up depth
- The diff between answers shows users where nuance lies
- Natural place to measure when fast answers are good enough
Costs
- Users may act on the first answer before the refinement lands
- Answers that change can feel unreliable if not explained
- Paying for two answers on every qualifying request
Worked example
An executive asks a strategy assistant a question about initiative performance.
- Executive“Which of our Q3 initiatives are behind plan?”
- Fast modelWithin 1.5 seconds, lists 3 initiatives whose status field is Red, labeled
First take, deeper check running. - Reasoning modelRuns in the background for 40 seconds. It compares milestones, spend and dates, not just status fields.
- Refined answerAppears below the first: “Two more are behind but still marked Green. Data Platform is 6 weeks late on its milestone and Partner Portal has spent 92% of budget at 60% scope.”
- Change noteThe refined answer says what changed and why: “Status fields lag reality; I used milestone and spend data.” The executive forwards the refined version.
Failure modes
| Failure | Symptom | Mitigation |
|---|---|---|
| Acting on the first take | Decisions made before the refinement arrives | Label the first answer clearly; block actions until refined when stakes are high |
| Unexplained flip-flops | Refined answer contradicts the first with no explanation | Always include a short "what changed" note |
| Refinement nobody sees | User left; background work wasted | Only refine when the user is still engaged or the request is high value |
| Redundant second answer | Refinement repeats the first answer at length | When they agree, show a one-line confirmation instead |