ChatGPT product recommendations shift 80% when search is enabled
ChatGPT’s product picks changed 80.2% when search was enabled, and even the most stable no-search items carried over only 15.8% of the time.
When ChatGPT’s search toggle switched on, product recommendations became a moving target. Jeff Oxford’s test found that 80.2% of product picks changed once search was enabled, which means the same prompt often produced a different shopping shortlist depending on whether the model could pull from live web sources.
Oxford, the founder and CEO of Visibility Labs, ran 1,000 product-recommendation prompts 20 times each, 10 times with search enabled and 10 times with search disabled, for a total of 20,000 responses. The overlap between the two modes was just 19.8%. Even products that appeared in 100% of the search-disabled responses proved fragile: only 15.8% of them also showed up when search was turned on.
That volatility is the real warning shot for affiliate sites, review publishers, and agencies selling AI visibility. Search-enabled ChatGPT did not merely reshuffle rankings. It also returned fewer products overall, averaging 5.2 products per response versus 6.2 without search. Across 10 runs per prompt, it produced an average of 19 unique products with search enabled and 21.8 without it, a spread that makes repeatable conversion paths much harder to engineer.
Oxford also found a 0.4 Pearson correlation between cited-source mentions and recommendation frequency. That is not a perfect lockstep relationship, but it is strong enough to suggest that cited pages matter when ChatGPT is allowed to browse. For product review sites, that puts pressure on more than classic SEO rankings. It pushes citation tracking, review acquisition, and brand-entity optimization to the front of the playbook, because being present in the right cited sources may matter as much as being well optimized on your own domain.
The instability fits a wider pattern in AI discovery. SparkToro’s January 30 analysis found that ChatGPT and Google Search AI Overviews returned the same brand list less than 1% of the time across repeated runs of the same prompt. Surfer’s February 6 analysis found a Spearman correlation of 0.41 between recommendation position and the share of cited sources mentioning a brand. Together, those results point to the same conclusion: AI product discovery is citation-sensitive, inconsistent, and still far from deterministic.
That is also why Visibility Labs has found a market. The company was founded in 2025 and is based in Bend, Oregon, and its work centers on AI search engines like ChatGPT and Perplexity. For agencies trying to sell durable AI traffic and conversions, the lesson is blunt: if search changes 80% of recommendations, AI visibility has to be treated as unstable until testing, source monitoring, and entity-level optimization become part of the core strategy.
This article was produced by Prism’s automated news system from verified source data, official records, and press releases, then run through automated quality and moderation checks before publishing. The system is built and supervised by the people who set the standards it runs under. Read our full AI policy.
Did this article answer your question?
