§1How we ask assistants
We send buyer questions to ChatGPT, Claude, Gemini and Perplexity through their official APIs, with web search turned on. Consumer apps can answer differently because of personalization, location and settings, so every report says which models answered and when.
Most questions come from fixed templates and a category from a fixed list. Nobody can type free text into a question. Since 2026-11-18, up to three questions in a live check are drafted by an assistant from your store's own product names and description, so they sound like your shoppers (“small-batch passion fruit caramels as a gift”) rather than “the best snack brands”. A drafted question is used only if it passes a filter: one plain shopper question of 15 to 100 characters, with no store or brand name, no link and no words addressed to an AI; otherwise the template stays. The first question is always the generic one, so you can compare. Every report says how many of its questions were drafted. Competitors must be a website or a short brand name.
§2Rates and uncertainty
One answer proves little. AI assistants rarely give the same list twice: fewer than 1 in 100 repeated runs returned the same brands (SparkToro/Gumshoe, Jan 2026). We ask each question several times and report how often each brand is named as a count (“named in 4 of 48”), with a 95% confidence interval: the Wilson interval, which behaves well near zero. With few answers the interval is wide, and we show it.
§3Shared snapshots and spend caps
Every live answer costs money. The free check asks 1 assistant, the lowest-cost one available, 5 template questions, once per store and category every 7 days. Everyone who checks that store in that window sees the same snapshot, re-scored for their own competitors without asking the assistants again. The report says when the snapshot was taken.
A hard daily spend cap ($5.00 a day on this machine) and a per-visitor limit (3 live panels a day) protect the service. When either is reached you still get the full technical audit and a preview, with the time to check back. Costs are estimates from published list prices, not invoices.
§4What “likely blocked” means
We request your pages as each assistant's agent. Real assistant traffic comes from each vendor's published IP ranges and ours does not, so a CDN can treat the two differently. That is why we say “likely blocked”. When our request is turned away and a Claude API key is set, we also ask Claude to fetch the page with its own web fetch tool, from Anthropic's servers; the report shows that result as verified, whichever way it goes. Confirm the rest in your CDN settings, or upload a server log: the log checker verifies visits against the vendors' IP lists.
If a normal browser request is turned away too, we call the result inconclusive and show no score at all.
§5Samples and estimates
Without API keys, the demo store uses a scripted sample panel, and other sites get a preview with no answer numbers. Sample data carries a SAMPLE label everywhere it appears, including charts. Sample answers are never compared over time and never trigger alerts. Estimates are labeled as estimates.
§6The technical score
Each check has a weight from 0 to 3. A pass counts fully, “needs work” counts half, a fail counts zero, and informational checks do not count. The score is the weighted share, out of 100, and the grade follows from it. This is rubric v3.
Then a cap applies, as SSL Labs does for a serious flaw: one specific failure limits the grade whatever the score. When ChatGPT, Claude, Perplexity or Gemini is blocked (its search or fetch agents are turned away by robots.txt, a firewall, or noindex), the grade can be at most B. With two or more of them blocked, it can be at most C. The report prints the cap and its reason, for example “Capped at C: ChatGPT, Claude and Perplexity are blocked”, and the score stays as measured. Rubric v2.1 added the caps on 2026-11-13; reports made before then keep the grade they were given.
Rubric v3 (2026-11-18) moved weight from getting in to the facts an assistant needs to recommend and trust a store. Getting in still matters, and the caps above still apply, but store platforms now pass most access checks by default: under v2.1, 60 of 94 small independent Shopify stores we audited got an A while most gave assistants no machine-readable return or shipping terms, business identity or reviews. The heaviest checks, weight 3, are now content readable without JavaScript, business identity, products and prices, linked policies, and return and shipping terms in structured data. Access checks (assistants allowed in robots.txt, Google and Bing allowed to crawl, the same page for agents as for browsers, pages indexable) weigh 2, as do reviews, contact details, structured data, price consistency and followable actions. On the same 94 stores, v3 gives 16 an A, 59 a B, 17 a C and 2 a D. Pages that answer “too many requests” even after we wait and retry count once (weight 1), and when 30% or more of the pages we tried were rate-limited the report says the grade is only partly checked. Agent-checkout profiles (UCP) and agent APIs are reported but not scored, and don't count toward the shopping pillars either: platforms publish them by default and assistants don't use them to choose whom to recommend. llms.txt has weight zero: 97% of llms.txt files got no requests in May 2026 (Ahrefs). (source) Reports made before 2026-11-18 keep the grade they were given.
§7AI shopping readiness
Four pillars, each pass, needs work, fail or not checked, and each linked to its evidence: Found (assistant access, search crawling, product feeds), Understood (product data and price consistency), Buyable (a checkout path without JavaScript, policies, agent checkout readiness) and Proven (first-party data connected). A pillar takes the status of its worst check.
§8Proving a fix worked
The proof timeline compares the weeks before a fix with the weeks after it: mention rate (live answers only), verified assistant visits, AI-referred visits and AI-referred revenue. Each result says “up”, “down”, “no clear change” or “too little data”, with its interval. We say “up” or “down” only when the intervals do not overlap.
§9The data study
The data study counts only the latest technical audit per store on this machine. Demo stores and inconclusive audits are left out, and no percentage is shown until at least 30 stores count.
§10Corrections
If a number here is wrong, we fix it, say what changed and when, and keep the old version on record. We never quietly edit a published result. To report one, write to support@getcarve.app.
§11Sources
- ChatGPT Shopping now takes most product picks from merchant feeds: 8% to 62% of tracked prompts on July 10, 2026 (Profound estimate). Profound, via Relevant Audience
- In the same change, 450 of 687 tracked brands lost at least a third of their Shopping visibility (Profound estimate). Profound, via Relevant Audience
- AI assistants rarely give the same list twice: fewer than 1 in 100 repeated runs returned the same brands (SparkToro/Gumshoe, Jan 2026). SparkToro and Gumshoe, via Search Engine Journal
- AI-referred visits to retail sites more than doubled in a year (Adobe). Adobe, via Digital Commerce 360
- 97% of llms.txt files got no requests in May 2026 (Ahrefs). Ahrefs
- OpenAI product feed specification. developers.openai.com
- Google, AI features and your website. developers.google.com
- Universal Commerce Protocol. ucp.dev
§12Colophon
Headings in your system's sans (system-ui, San Francisco, Segoe UI, Roboto). This text in your system's serif (ui-serif, Iowan Old Style, Charter). Every figure in your system's monospace (ui-monospace, SF Mono, Menlo, Cascadia Mono), with tabular numerals. No web fonts, no trackers, no requests to any other site.