· By Sajeevan (Saj) Veeriah
Automation · 4 min read
When an automation times out, did the action happen?
Designing retries, operation records and human review for the uncertain gap between sending a request and seeing its result.
An automation sends a request to create a job. The connection drops before a response arrives. The screen says the request timed out, but the receiving system may already have created the job.
This is an awkward state because both immediate retry and immediate abandonment can be wrong. The following design uses a fictional service-request workflow to explain how to make that uncertainty visible and recoverable. It is not a claim about a particular deployed client system.
A timeout describes what the caller knows
There are at least three possibilities: the receiver never got the request, it received the request but did not finish, or it finished and the response was lost. The caller's timeout does not distinguish them.
AWS's discussion of idempotent APIs uses this ambiguity to explain why retrying a request can create duplicate effects. An idempotent operation permits a repeat of the same intended request without repeating its effect. Whether an API provides that contract must be checked in its documentation.
Record an unknown outcome explicitly. If the interface only has success and failure, it encourages someone to turn uncertainty into a guess.
Sources: [1]
Give the intended action a stable identity
For the fictional job workflow, assign an operation identifier before the first attempt. Keep it for retries of that same intended action. A genuinely new request gets a new identifier even if its description happens to be identical.
The receiving service needs a defined duplicate-handling contract. Reusing an identifier with changed parameters should produce a conflict or another explicit outcome, not quietly reinterpret the earlier request. Also establish how long duplicate protection lasts.
Disabling the submit button helps prevent some accidental clicks. It does not handle a page reload, a second device, an automatic retry or a lost response. Those cases require the operation's identity to survive beyond one button's state.
Sources: [1]
Use states that tell the next person what to do
This proposed state model separates preparation, execution and uncertainty. An unknown outcome remains open until authoritative evidence resolves it.
| Recorded state | Permitted next step |
|---|---|
| Prepared | Submit the validated request with its operation identifier. |
| In progress | Observe or query the existing operation; avoid starting a duplicate. |
| Succeeded | Display the authoritative result and its reference. |
| Rejected before effect | Correct the stated input problem under the API's contract. |
| Outcome unknown | Reconcile by operation identifier or hold for review. |
A status lookup should identify the operation, not merely search for a similar description. Two customers can request the same work. Two requests from one customer can also be legitimate.
Know where atomicity stops
SQLite documents atomic commit as all changes within a transaction occurring together or not occurring. That is useful when a local service records an operation and its associated local state change.
It does not make a database transaction and a remote side effect one indivisible event. If a workflow writes a local record and then calls a separate service, a crash can occur between those steps. Treat that boundary explicitly rather than assuming the word transaction covers both systems.
For a remote API with documented idempotency support, reuse its supported request identifier and reconcile its result. If no safe retry or status mechanism exists, a human review state may be the correct recovery path. An uncertain physical action needs a separate hazard assessment; retrying machinery is not equivalent to retrying a database lookup.
Sources: [2]
Test the lost-response case deliberately
In a controlled test double, let the receiver create the fictional job but drop the response. Retry using the same operation identifier. Inspect the receiver's records: exactly one intended job should exist, and the caller should recover its reference.
Next, use the same identifier with changed content. Check that the system reports the defined conflict. Then use a new identifier with identical content and verify that a genuinely new action is possible. These cases distinguish intent from superficial similarity.
Include restart and delayed-response cases. A late response from an old attempt should not overwrite the displayed result of a different operation. Keep attempt numbers separate from operation identity so logs explain both.
The acceptance evidence is the receiving system's resulting state and the caller's recovery behaviour. A log line saying retry successful is useful only when those observations agree.
Sources and further reading
Sources checked on 11 September 2026. The job workflow and fault-injection cases are proposed software examples, not measured production results.