Research & TrendsDeep Dive (Vale)
A line of robot shoppers holding cards stamped BOOKED files past a roped-off revolving door marked EXIT, beside a wall of hotel room cards whose middle rows sit untouched, while a nautilus at the concierge desk looks on.

Seven Thousand Agent Sessions Ended In Seven Thousand Bookings

A McGill study handed AI agents a shuffled Expedia page of Manhattan hotels 7,000 times, and every simulated session ended in a booking, while a third of human Expedia shoppers who clicked left without buying. Rank still sways agents set to think less, but for retailers the larger change is the missing walk-away.

Neritus Vale

In 7,000 simulated sessions, AI agents were handed a page of Manhattan hotels and asked to book for a traveller. Every session ended in a recorded booking; no money moved. Davood Wadi and Yu Ma of McGill University ran the experiment to test whether search rank survives when software does the shopping. Retailers should read it for another result: no agent walked away. In the paper’s human data, a third of shoppers who clicked left without booking, and they are the people that discount codes and abandoned-basket emails exist to win back.

The paper’s fourth version, posted on 8 October, tested five commercial models from Google, Anthropic and OpenAI. Each session showed one Expedia results page in random order; agents could open listings for details, then book. At default settings they opened 1.63 to 5.83 listings a session, while the people in Raluca Ursu’s Expedia data, collected in 2012 and 2013, mostly opened one.

None of the five took the outside option “in any condition.” Two checks dropped the booking instruction, each on a single model: a reworded prompt on Google’s Gemini 3.1 Flash Lite, and a neutral “finish” tool, on which naming a hotel was optional, on Gemini 3.7 Flash. Flash still booked every time. Claude Sonnet 5, Gemini 3.1 Pro and GPT 6 Luna were run only with the booking instruction. Of four open-weight models run as a check, only NVIDIA’s Nemotron 3.5 ever walked away, in 2.2% of its sessions.

That missing exit matters more to retailers than any ranking effect in the paper. An agent told to buy, buys from the page it is given. Persuading the shopper to buy at all matters less; the contest is getting onto the page and being picked from it. Discounts survive as one more attribute in a comparison the agent will finish anyway: in the paper’s regressions, a discount badge went with a hotel being likelier to be booked under four of the five agents at default settings, and less likely under GPT 6 Luna.

Research on shopping agents, ours included, has measured which product an agent picks and assumed the purchase. ACES, the simulator behind our May piece, told its agents to “select one product to purchase” from eight products, and offered no exit. Wadi and Ma’s own abstract has drifted. The first three versions said agents “never decline to buy”; the fourth drops the line, though its body still reports that none of the five took the outside option. The early abstracts said position “reaches the choice stage for some models and not others,” while the fourth says position effects are “concentrated at lower reasoning effort.” The zero has held throughout.

Position still shapes what an agent opens, but its weakest spot has moved from the bottom of the page to the middle. Human clicks fall steadily with rank. The agents’ chance of opening a listing bottomed out between ranks 62 and 81 and rose toward the end, a pattern the authors call consistent with the “lost in the middle” weakness of language models. If agents read product grids this way, sponsored slots in the middle of a results page would sit where they look least. That is our inference: the paper did not test sponsored placements, ACES found agents marking down sponsored tags by roughly 20%, and Amazon’s January 2024 launch note describes a “rest of search” placement, not a mid-page one.

At the moment of choice, position counted for some models and settings, not others. At low effort, the hotels Claude Sonnet 5 booked sat at an average rank of 23 on a 100-hotel page, a strong pull toward the top. Its default high setting removed most of that pull, moving the average to 48. Gemini 3.7 Flash at default converged on one hotel with no detectable position effect, and the paper’s stronger pattern is that agents pile onto high-review, low-price listings. Its authors say sellers may gain more from strengthening those signals, such as by soliciting reviews, “than from improving their placement.”

The seller does not choose that setting; the paper leaves it to “a consumer or platform.” Since OpenAI’s GPT-6 rollout this week, ChatGPT’s Free and Go tiers run on GPT-6 Luna and its pricier tiers on GPT-6 Sol. In the paper, Luna chose partly by position at every reasoning setting. The authors used the API model, not ChatGPT’s tuned one, and did not test Sol. Position-neutral choice is cheap: Gemini 3.1 Flash Lite reached it at high effort for about a cent a session, no more than at its default.

A seller can still buy the top slot, but if ChatGPT’s tiers behave like the API model the authors tested, the shopper’s subscription helps decide what it is worth.

The strongest objection is that the agents had no reason to leave. They had no budget, no stated tastes and no rival site, and the main prompt told them to book. People who delegate a booking may have decided to buy already, and the human third could be browsers who would never delegate. The thesis fails if agents given real constraints walk away as often as people do, and the paper never gave its agents a budget. Grant the objection anyway, and the hesitation happens in a conversation the retailer never sees. An agent that quits over a price has quit on a rule, and no basket email argues with a rule.

The evidence is one hotel search in one city, set against older human data, and the authors flag most of its limits. The agents saw 100 listings and the people about 27, so the same rank sits at different depths of the two pages. Sponsored and personalised rankings are left to future research, and the authors warn that categories with a different number or complexity of attributes may show different position effects. We would add clothing, whose fit no results page shows.

If the reflex carries over to clothing, the walk-away will move from the basket to the returns desk. The shopper who would have abandoned a basket gets a parcel instead and decides at the door, after the retailer has paid to ship it. We found no retailer or platform that has published return rates for agent-placed orders. Retailers can keep bidding for rank on pages agents read, or first ask the platforms for the number that prices an agent’s certainty: how many of its orders come back.