Before an agent retries a tool call, classify the prior operation
Absent, running, completed or uncertain: the safe recovery action for each state, and how to test it.
When an agent worker fails, a retry can send the same message twice or start a second job while the first still runs. The question is what the system knows about the prior operation before it retries.
The short answer
- Classify first. Before a retry, decide whether the prior operation is absent, still running, completed with a saved result, or uncertain.
- Use evidence, not elapsed time. A worker timeout does not prove that its tool stopped. A missing result does not prove that the tool had no effect.
- Keep one identifier. Give each logical action an operation identifier that stays stable across recovery attempts, and pass it to the tool when its API supports idempotency keys.
Safe recovery action by observed state
| Observed state | Safe recovery action |
|---|---|
| Confirmed absent | Start the operation. |
| Confirmed running | Continue to observe the existing operation. |
| Completed with a saved result | Return the saved result without calling the tool. |
| Outcome uncertain | Reconcile with the tool's external system, then decide. |
When the answer changes
- A saved result is not a saved side effect. A completed-result cache stores return values only after the system saved them. If a tool produces a side effect and fails before its result is cached, a retry can repeat the side effect.
- Uncertain means reconcile. Look up the action by operation identifier or external receipt. If the external system cannot answer and a duplicate would be harmful, stop and request a decision.
- Our test was narrow. In a controlled production fault on September 7, 2026 with one synthetic agent, a saved shell result was reused once. We did not test the uncertain window, and the test does not show exactly-once execution.
The full post has the fault tests to run on both sides of result persistence.