Research & TrendsDeep Dive (Vale)
A nautilus in a naval coat stands between a loading dock stacked with plain crates labelled Solr, AlloyDB, ScaNN and E5 and an open safe in the back office of a Target store, which holds a ledger of receipts and a brass dial beside a card marked withheld.

Target Published Its Search Stack and Kept the Weights

In Target's live tests, an off-the-shelf embedding model left search demand per visitor 7.14% below keyword search alone, while a model trained on Target's own click, cart and order data lifted it 1.10%. The paper names the parts a retailer can buy and withholds the label and merge settings learned from Target's shoppers.

Admiral Neritus Vale

For four weeks, Target added an off-the-shelf embedding model to its keyword search on live traffic, and demand per visitor (order value divided by visitors) came in 7.14% below keyword search alone. A later model, trained on a record of what Target’s own shoppers click, add to cart and buy, and merged into the keyword results by weights tuned on Target’s traffic, came in 1.10% above. The comparison bundles several changes, and the paper does not say which merge rule the off-the-shelf test used. The paper reporting both tests, submitted to arXiv on 25 September, names the parts a retailer can buy and withholds the settings learned from those shoppers: the label weights and thresholds, and the merge weights.

In Retail Product Search: A Practical Approach at Target, Darshan Sonagara, Qujiaheng Zhang, Ankit Singh and Alex Li, of Target’s Data Sciences group, describe a system built mostly from parts anyone can download or rent. Apache Solr, an open-source engine, handles keyword search. Vectors for retrieval by meaning sit in Google’s AlloyDB database, indexed with ScaNN, an AlloyDB index type. The encoder that turns queries and product text into those vectors starts from e5-small-v2, free to download under an MIT licence and small enough to run on commodity CPU servers. The paper’s figure for zero-result searches, a cut of about 50%, comes from the deployed fine-tuned model’s test.

The paper’s confidentiality notes list what Target kept back. Target reports only relative lifts, “following company policy”, and withholds the settings that turn its logs into training labels. The logs, aggregated by query and product rather than by person, were sampled down in dominant categories such as home and apparel to make room for smaller ones, then filtered into about 20 million labelled pairs. A pair counted as wanted only if its weekly score, which weights orders above add-to-carts and add-to-carts above clicks, held above a threshold for several weeks. Products shown for a query at least a set number of times without a single click, cart or order became negatives: in the paper’s words, “shown often and consistently ignored”.

Nothing on the paper’s parts list has seen what Target’s shoppers chose and what they ignored.

The rule that merges the two result lists changed what the first fine-tuned model did to orders. With reciprocal rank fusion (RRF), which promotes products that both lists rank, the results were mixed: demand per visitor rose 0.49% and add-to-carts rose, but clicks fell and order conversion dropped 1.20%. The authors’ explanation is that the two lists overlap little, so the agreement RRF rewards was rarely there. Weighted interleaving fills each slot from one list or the other in proportion to a set weight. Target tuned those weights through successive A/B tests, starting from an even split, and withholds the values it settled on.

Elastic and Google both document RRF as a way to merge keyword and vector results. Elastic’s documentation says RRF “requires no tuning” and removes “the need to figure out what the appropriate weighting is using linear combination” of scores. Google’s AlloyDB documentation, for the database that holds Target’s vectors, describes hybrid search as separate vector and text searches, “then combining and re-ranking results using Reciprocal Rank Fusion (RRF)”. Its sample query uses the constant of 60 that Target used in its own RRF test. Elastic’s claim concerns blending scores, a different problem from the channel shares Target tuned. Target’s tests set RRF against interleaving, and interleaving did better on order conversion; the paper does not compare tuned weights with an even split.

The first model trained on Target’s records already moved demand about as far as the final one did. Interleaved with the keyword results, it lifted demand per visitor by 1.11% and order conversion by 0.74%. The deployed model then added expanded training data, hard negatives, field markers and product highlights, among other changes. Clicks and conversion rose further, while demand per visitor stayed level.

Vector search also needed a filter that knows Target’s catalogue. A precision-control module removes vector results that contradict the gender, size or brand named in a query, or the item type and category a query classifier predicts from it. Without filters, precision at ten was 0.836, so roughly one product in six in the vector channel’s top ten was not a desirable result. With both sets of filters, precision rose to 0.885, or about one in nine. The paper’s examples come from bath towels, groceries and pet products, and it reports no apparel-only result; in clothing, where gender and size decide whether a result can be worn at all, the filters should matter more. Each filter’s threshold was tuned before launch, another setting the paper leaves out.

The strongest objection is that the e5 models Target tested are older, and that a newer, larger model might not need Target’s data. For that to hold, a generic model would have to rank Target’s products as Target’s shoppers would, without ever seeing their choices. Target’s own tests give little reason to expect size to close that gap. On queries graded by human annotators for topical relevance, the larger e5-large-v2 beat the small model by 0.0068 on NDCG, a standard ranking score. The authors trace the live loss to a model “trained for general semantic similarity rather than for the products users choose to buy”. If generic models keep improving, the zero-shot loss may shrink, but the choices will stay in Target’s logs.

For a retailer buying AI search, the decision that matters is who learns the settings and who keeps them. A vendor cannot ship the data, but it can learn the settings on the retailer’s traffic. Target’s off-the-shelf test shows what the system does before anyone has. If models and vector databases keep getting cheaper, the settings will be where one retailer’s search differs from another’s. The clause to read in the contract is the one that says whether the labels, thresholds and merge weights stay with the retailer whose shoppers produced them.