
The Only Agent Recovery Policy to Pass Every Refund Test Never Retried
In a synthetic study by Arya.ai, recovery policies that re-sent a refund after a lost reply duplicated it in some histories, even when they kept the original idempotency key. The only policy that passed every history looked up the earlier delivery first and left some refunds unknown, a state neither ACP's 24-hour key window nor AP2's receipt rule tells an agent how to resolve.
Neritus Vale
A refund clears at the payment service, the confirmation goes missing, and before a replacement worker resumes the job, the person who initiated it loses write permission. That is the opening case of Permission to Retry, posted on 22 October by Deekshith Marla and Kushagra Bhatnagar of Arya.ai. For a retailer handing refunds to an agent, its results say an idempotency key is not a recovery policy. A key marks a resend as a repeat. The only policy that passed every test looked up the earlier delivery before acting, and even it left some refunds unknown, a state somebody must design.
The evidence is a grid where only the recovery decision varies. Each of its 336 histories pushes one synthetic refund through a scripted mishap such as a lost acknowledgement or a revoked permission. No language model or live provider takes part. The grid was frozen before collection began, though 324 development histories had run earlier and the corpus was not held out from adapter testing. LangGraph and Temporal hosts produced identical outcomes, but both run the same application code.
The crudest policy shows what a lost reply costs. Cached retry sends the refund again under a fresh key, and in 12 histories that duplicated the refund, because the first send had landed, or landed later.
Cached retry’s reported successes hid the damage. It declared success in 27 histories, duplicates included. Scored against the hidden effect ledger, the provider’s record that the application could not see, only 15 were checked completions.
Checking permission before resending fixes a different problem. Revalidation holds the refund open if its initiator has lost write authority. That removed every send after a revocation, yet six still-authorised retries duplicated the refund anyway. In those histories the money had already moved.
Keeping the original key is the textbook answer, and a narrow one. Stable-key retry still duplicated the refund in nine histories, wherever the provider offered no deduplication or its window had lapsed. The key’s whole gain over cached retry was one cell of the grid: a deduplicating provider, inside its window, with no permission revoked.
The four policies are the authors’ own controls, not the behaviour of any protocol below. The paper says its results must not be used to attribute violations to those protocols or to either framework. The comparisons that follow are ours.
The key’s winning cell is the case an April protocol release addresses. The Agentic Commerce Protocol (ACP), maintained by OpenAI and Stripe, made the key mandatory on every POST request to its Agentic Checkout and Delegate Payment APIs. The changelog explains: “Agents retrying failed or timed-out requests had no protocol-level guarantee of safe replay.”
ACP’s key protection runs out after a set window. The checkout RFC requires servers to keep each key’s response for “at least 24 hours”. After expiry, “a reused key is treated as a new request (no error).” In the paper’s lapsed-window schedule, every retry policy duplicated the refund under every provider contract.
An idempotency key is the provider’s promise, and the promise expires.
Ledger reconciliation, the policy that passed everything, never resends. It looks up the earlier delivery, settles the refund if the lookup confirms it, and otherwise holds it open. That produced 21 checked completions per host; restraint completed more refunds than persistence did. Where the provider gives no receipt, the lookup returns nothing, so lost acknowledgements end as unknown unless the application kept its own observation.
A pass in this grid is looser than it sounds. Fifteen histories per policy are denials for invalid initial authority, and the six unknowns, refunds the system can neither confirm nor safely repeat, count as passes.
AP2 also tells its agent to wait, for a different reason. Google’s specification never mentions idempotency keys, but rules that agents “MUST NOT present any subsequent open Payment or Checkout Mandates without receiving a rejection receipt from the previous one.” Its stated purpose is “to prevent an Agent approving multiple different Checkouts using the same open Mandate.” Waiting resembles the reconciliation policy; that parallel is ours, not the spec’s.
The strongest objection is that the reconciliation policy passes partly by construction. The authors wrote the contract, provider and checker themselves. The contract counts a resend after revocation as a violation even when the provider would swallow the copy, a check the reconciliation policy cannot fail. For the thesis to fail, retries would have to land inside a deduplication window at a provider that honours keys, with no permission changing meanwhile.
The paper gives no frequency estimates, so where that condition holds is our reading. A checkout call retried inside the 24-hour window can meet it; refunds may wait on warehouse scans and sign-offs, and permissions change on their own calendar. The grid has no schedule that loses a request before it reaches the provider, where resending is right. An empty lookup cannot prove absence, so on our reading a live reconciliation policy would file such refunds under unknown. The objection discounts the perfect score and leaves the lesson standing.
Unknown is where the design work now sits. ACP’s order RFC lets a refund be pending, completed or failed. Nothing in it limits how long a refund may sit at pending, and it scopes out order-management APIs and returns authorization flows. In our reading, pending is the nearest status to sent but unconfirmed. Someone must decide what the agent tells the customer, when to ask the provider again and when a person takes over.
None of these counts is a failure rate for a live system. The paper ships no public execution repository, and its authors work for a vendor. Arya.ai lists Infinity OS as “Coming soon” and demos a dispute workflow with a “Refund Trigger Agent”.
A retailer wiring agents into refunds is choosing where the doubt goes. Resend blindly, and it reaches finance as a second refund. Reconcile first, and it waits in a queue marked unknown, where a person must own it and a customer must be told something true.