How AI assistants see a Shopify store
Short answer
AI assistants reach a Shopify store through search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot), through fetches made for a user’s question, which OpenAI and Perplexity say may not follow robots.txt, and, for Google’s AI Overviews and AI Mode, through the normal Google index. Training crawlers such as GPTBot and ClaudeBot are separate. Blocking a search crawler can keep the store out of that assistant’s search answers.
- Google’s AI Overviews and AI Mode have no extra technical requirements: a page must be indexed and eligible to be shown with a snippet.
- OAI-SearchBot surfaces sites in ChatGPT search; GPTBot crawls content that may be used for training; the two are controlled separately in robots.txt.
- Anthropic separates ClaudeBot (training), Claude-SearchBot (search) and Claude-User (fetches for a user’s question).
- ChatGPT-User and Perplexity-User act on a user’s request: OpenAI says robots.txt rules may not apply to them, and Perplexity says its fetcher generally ignores robots.txt. Anthropic says its bots, including Claude-User, honor robots.txt.
- Shopify stores can edit robots.txt through a robots.txt.liquid template; Shopify calls it an unsupported customization that can cause a loss of all traffic if done wrongly.
- On Shopify, blocking AI crawlers in robots.txt doesn’t stop Shopify Catalog from sending product data to agentic storefronts the merchant has activated.
Some shoppers now ask an assistant before they search. For a merchant, the first question isn’t how to be recommended; it is whether the assistant can reach the store at all, and what decides that. The answer is less mysterious than the marketing around it suggests.
Four ways an assistant reaches your store
- Its own search crawler. OpenAI, Anthropic and Perplexity run crawlers that build the index their assistants search: OAI-SearchBot, Claude-SearchBot and PerplexityBot.
- A live fetch for one user. When someone asks a question, the assistant may visit a page there and then, as ChatGPT-User, Claude-User or Perplexity-User. Because a person started the request, OpenAI says robots.txt rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them. Anthropic says Claude-User honors robots.txt.
- Google’s index. AI Overviews and AI Mode draw on Google Search. Google says there are no additional technical requirements: a page must be indexed and eligible to be shown with a snippet.
- Shopify Catalog. If you sell through agentic storefronts such as ChatGPT or Microsoft Copilot, Shopify shares your product data through Shopify Catalog, independently of robots.txt. That sharing is managed in your agentic storefront settings, not by crawler rules.
Separate from all four are training crawlers, such as GPTBot and ClaudeBot. They collect content that may be used to train models. Blocking them is a decision about training, not about whether you appear in search answers.
The crawlers, and what blocking each one does
| User agent | Operator | Purpose | If you block it |
|---|---|---|---|
| OAI-SearchBot | OpenAI | Surfaces sites in ChatGPT search | The site isn’t shown in ChatGPT search answers, though it can still appear as a navigational link |
| GPTBot | OpenAI | Crawls content that may be used to train models | Signals the content shouldn’t be used for training |
| ChatGPT-User | OpenAI | Visits pages for a user’s request | robots.txt rules may not apply |
| Claude-SearchBot | Anthropic | Indexes content to improve search results | May reduce visibility and accuracy in Claude’s search results |
| ClaudeBot | Anthropic | Collects content that could contribute to training | Signals that future content should be excluded from training data |
| Claude-User | Anthropic | Fetches pages for a user’s question | Claude can’t retrieve the page for a user, which may reduce visibility in user-directed search |
| PerplexityBot | Perplexity | Surfaces and links sites in Perplexity search; not used for training foundation models | Perplexity recommends allowing it for the site to appear in its search results |
| Perplexity-User | Perplexity | Visits pages for a user’s question | Perplexity says it generally ignores robots.txt, since a user requested the fetch |
| Google-Extended | Limits AI training and grounding in some of Google’s other systems | To limit what Google Search shows, including AI features, Google points to nosnippet, max-snippet and noindex instead |
What controls this on Shopify
Every Shopify store serves a default robots.txt. It allows public content and blocks pages such as the admin, cart, checkout and filtered or sorted collection URLs that duplicate content. Bot management at the network level is handled by Shopify, with no action needed from the merchant.
You can change it by adding a robots.txt.liquid template to the theme, to allow or disallow URLs, add sitemap URLs or block specific crawlers. Shopify is blunt about the risk: it is an unsupported customization, Shopify Support can’t help with it, and incorrect use “can result in loss of all traffic”. Shopify also recommends editing with Liquid rather than pasting plain text, so the store keeps receiving Shopify’s own updates to the default rules.
Two limits are worth knowing. Robots.txt rules are advisory: Shopify notes they aren’t guaranteed to be honored by every crawler. And blocking AI crawlers doesn’t stop Shopify Catalog from sending product data to agentic storefronts you have activated.
Shopify’s default template loops over robots.default_groups, the default rules, which Shopify updates regularly; keeping that loop is how Shopify recommends preserving those updates. A rule that isn’t part of a default group goes outside that loop. This example keeps every default rule and opts out of OpenAI’s training crawler only, leaving OAI-SearchBot, and so ChatGPT search, untouched:
{% for group in robots.default_groups %}
{{- group.user_agent -}}
{% for rule in group.rules %}
{{- rule -}}
{% endfor %}
{%- if group.sitemap != blank -%}
{{ group.sitemap }}
{%- endif -%}
{% endfor %}
User-agent: GPTBot
Disallow: /Whether to opt out of training at all is a business decision, not a technical one. The point of the example is the separation: a training opt-out and a search opt-out are different lines.
Before changing anything, open yourstore.com/robots.txt and read what it says today. If a theme or developer added rules in the past, check that none of them block a search crawler you want to keep. Shopify also notes that a store with a robots.txt.liquid template can still serve rules the current default file has dropped, such as Disallow: /search; deleting the template restores the current default.
What they can read once they arrive
Access is only the first step. Google’s guidance for its AI features is the same as for Search: allow crawling in robots.txt and at the CDN or hosting layer, link pages internally, offer a good page experience, and make important content available as text. That last point matters on stores, where key facts often live only in images, sliders or app widgets.
- Product facts in text. Materials, sizes, compatibility and care instructions written on the product page, not only in images.
- Answers where the question is asked. Shipping, returns and sizing questions answered on the pages where shoppers have them, in plain sentences.
- Consistent names. The same product and brand names across product pages, collections and policies.
- No snippet blocks on pages you want used.
nosnippet,max-snippetandnoindexlimit how Google can show a page, including in AI features. - Structured data that matches the page. Google lists structured data that matches the visible text among its best practices, and says no special schema is needed for AI features. It is ordinary SEO, not a separate AI tactic.
Checking where you stand
- Read your live robots.txt and list every rule that names a crawler.
- In Search Console, AI Overviews and AI Mode traffic is included in the Performance report under the “Web” search type.
- Ask the assistants your customers use the questions they would ask, and note whether and how the store appears. It is a manual spot check, not a measurement.
- Fix what you control: crawl access, text content, consistent facts. Then check again later.
What nobody can promise
Each assistant decides for itself which sources to use, and those systems change often. No one can guarantee that a store will be cited or recommended. What a merchant can do is make sure the store isn’t shut out by accident and that its pages state the facts clearly enough to be quoted correctly.
That is the scope of Lintel’s generative engine optimization work, and it rests on the same foundations as performance and technical SEO.
Questions
- Can I control AI crawlers on a Shopify store?
- Yes, by adding a robots.txt.liquid template that allows or blocks specific user agents. Shopify calls it an unsupported customization, and the rules are advisory. Product data shared through Shopify Catalog is managed separately, in your agentic storefront settings.
- If I block GPTBot, will my store disappear from ChatGPT?
- Not from ChatGPT search. GPTBot is OpenAI’s training crawler; OAI-SearchBot is the one that surfaces sites in ChatGPT search, and they are controlled separately.
- Do I need an llms.txt file or special schema for AI search?
- Not for Google. Google says no AI text files, special markup or schema.org structured data are needed to appear in AI Overviews or AI Mode. Pages must be indexed and eligible for a snippet.
- Can anyone guarantee my store appears in AI answers?
- No. Each assistant chooses its own sources. You can remove technical barriers and state facts clearly; the choice to cite stays with the system.
