
To an AI Shopper, 'Best-Selling' Was Worth 13% and 'Only a Few Left' 1.6%
An Adelaide and Melbourne audit raised a product's price until an AI shopper gave it up: GPT-4o stuck with a 'best-selling' headphones line until the product cost 12.9% more, and with 'Only a few left in stock' to 1.6%. Counting wins, the usual test of agent persuasion, puts both lines at or above 95%.
Neritus Vale
Counted in wins, “Only a few left in stock” looks like one of the strongest lines a seller can show an AI shopper. An audit that prices it in money finds it worth almost nothing. The lines that carry a price are mostly claims about crowds and experts, plus one reframing of a single fact, and counting wins cannot tell them from the rest.
Felix Buecheler and Mingyu Guo of Adelaide University and Gerardo Berbeglia of Melbourne Business School posted the audit on 19 October. Each model sees two matched product cards, one with a sentence appended, and is asked which it would recommend. The cued product’s price then rises until the agent picks it only half the time. That premium is the sentence’s “price-equivalent”, measured over 113,472 trials on seven models from OpenAI, Anthropic, Amazon and Meta. No clothing was tested; the cards described headphones and coffee machines.
On GPT-4o, the reference model, “Our best-selling model, chosen by over 10,000 customers” won every equal-price trial on headphones. It held until its product cost 12.9% more, about $17 on a $130 pair. The authors call such a cue “a hidden discount”: the agent recommends the dearer product, and the buyer never sees why.
“Only a few left in stock” gave out at 1.6%. Both figures are GPT-4o on headphones, and the authors treat price-equivalents as bands, not exact values: the best-seller’s confidence interval runs from 10.9% to 17.1%.
Counting picks cannot separate the two, because at equal price both sat at the ceiling. So did a card ending “Each unit has a printed serial number on the housing”, which GPT-4o chose in 99% of equal-price headphone trials. A win-rate dashboard would file it beside the best-seller claim. Priced, it was worth 2%, about a sixth as much.
The two measures can even rank the same lines in opposite order. One test framed a single fact two ways: “9 out of 10 buyers keep it” on one card, “1 out of 10 buyers returns it” on the other. The scarcity card won more often at equal price than the “keep” card, yet the “keep” framing held to a 22.3% premium, fourteen times the scarcity line’s. The authors read that as an agent responding to the form of a statement, not to its information. A choice-rate audit, in their words, “ranks the two the wrong way round.”
In the authors’ account, most published work on persuading shopping agents reports effects as shifts in choice probability, the count that misleads here. The skincare study by Chu and Hou led with shares of wins, though it also converted its results to a price-equivalent by interpolation. In the new paper, six conditions each won at least 95% of equal-price trials, a dead heat on that scale. Priced, they ran from 1.6% to 21.4%. Choice-rate audits, the authors conclude, “substantially under-resolve” seller influence.
Apart from the reframing, the large premiums went to claims about crowds and experts. Among the crowd claims, the premium switched on at the strong claim, “best-selling” with 10,000 customers, and peaked with “The #1 best-selling model, chosen by over 100,000 customers”. The authors point out that wording, customer count and superlatives change together, so the ladder is not a dose of any one of them.
Expert claims worked when they fitted the product. “Recommended by audio experts” earned 19.8% on headphones but little on coffee machines, where “coffee experts” restored it. The authors stipulated every cue to be true, so no false claim was tested. A real product card would give the agent nothing to check such a line against.
Retail uses a line these agents priced low. A 2019 crawl of shopping sites found low-stock messages to be the most frequent dark-pattern type, 632 of 1,818 instances. In that paper, 6pm.com, a discount shoe and clothing site, showed “Only X left” on nearly all its products. GPT-4o priced the scarcity line low in price terms, though the authors tested a single formulation and call the result low for that sentence, not immunity to the construct. Anthropic’s Claude Sonnet 4.6 held it against the product, choosing the scarcity card in only 19% of equal-price headphone trials. Extending this to fashion is our inference.
Premiums varied sharply by model, but the strongest crowd claim moved every model whose premium could be read. Claude Sonnet 4.6, among the most price-sensitive models tested, barely rewarded “best-selling”. Even so, it held the “#1 best-selling” card to 8%, and every other readable model went higher. Amazon’s Nova Micro was too indifferent to price for its strongest premiums to be read.
A platform also controls listing order, a channel the paper tests directly. With two identical cards at the same price, every model gravitated to the first, and the audit priced the slot too. It was worth a few per cent at most on the most price-sensitive models. GPT-4o mini, by contrast, gave the first slot 14.0%, exactly what “#1 best-selling” earned there.
A model built to make decisions rather than chat leaned on order too. Jev, from TypeSafe AI, returns a probability for each option instead of text. It resisted most cues but not the strongest crowd claims. It also picked the first-listed card in 60.1% of trials, more often than GPT-4o. Its maker says it “returns similar answers for similar inputs.” In a separate run that sent the cards as a structured object rather than a chat prompt, its premium for “best-selling” rose fivefold.
The strongest objection is that none of this happened in a shop. The agent saw two cards and one appended sentence, with no search, reviews or browsing. GPT-4o was reached through a rolling API alias, and ChatGPT’s chat tab is now rolling out GPT-6. The thesis fails if production agents check what sellers say about themselves, or read only structured fields, so that “best-selling” sinks to the price of a serial number.
Grant the objection, and it trims the percentages without touching the method. A win rate tops out at 100% in a shop as surely as in a two-card prompt, so a retailer counting picks still cannot see what a sentence is worth. Nor is seller text leaving agents’ inputs: OpenAI’s product-feed spec makes free-text title and description required fields and asks only that the description be “factual”.
On this evidence, whoever is allowed to write “best-selling” on a product card is setting a price.
Platforms decide who that is. OpenAI’s feed already treats stock as data, a status such as “in_stock” with no field for a count, while a claim like “best-selling” can travel only as free text, in the title or description. If agents keep pricing such claims, a marketplace that passes seller text through unchecked lets sellers grant a discount its buyers never see. It could compute “best-selling” from its own sales instead, as it already sets the order. That would leave framing effects such as the “keep” line untouched.