AI Bots We Check For
We track two tiers of AI bots because they influence ecommerce performance in different ways: shopping agents can directly drive transactions, while crawlers and search bots shape how your catalog appears in AI discovery.
AI Shopping Agents
These bots browse stores and can complete shopping steps on behalf of users, so allowing them can directly impact assisted commerce conversion.
| Bot Name | Operator | Description |
|---|---|---|
| Operator | OpenAI | Autonomous shopping agent by OpenAI |
| ChatGPT Agent | OpenAI | Browses websites on behalf of ChatGPT users |
| AmazonBuyForMe | Amazon | Purchases products on behalf of Amazon customers |
| NovaAct | Amazon | Amazon's browser automation agent |
| GoogleAgent-Mariner | Google's web browsing AI agent |
AI Search Indexers & Retrieval
These bots index or retrieve content for AI search and answer products, so allowing them improves discoverability in AI assistants.
| Bot Name | Operator | Description |
|---|---|---|
| OAI-SearchBot | OpenAI | Indexes content for SearchGPT results |
| ChatGPT-User | OpenAI | Fetches pages when ChatGPT users ask questions |
| Claude-User | Anthropic | Fetches pages when Claude users ask questions |
| Claude-SearchBot | Anthropic | Indexes content for Claude search results |
| Gemini-Deep-Research | Fetches pages for Gemini's deep research feature | |
| PerplexityBot | Perplexity | Indexes content for Perplexity search |
| Perplexity-User | Perplexity | Fetches pages when Perplexity users ask questions |
| Amazonbot | Amazon | Crawls sites for Alexa answers |
| meta-externalfetcher | Meta | Fetches pages for Meta AI responses |
AI Training Crawlers
These bots collect content for model training datasets, so many merchants treat them separately from search-facing bots.
| Bot Name | Operator | Description |
|---|---|---|
| GPTBot | OpenAI | Crawls sites to train OpenAI models |
| ClaudeBot | Anthropic | Crawls sites to train Anthropic's models |
| Google-Extended | Crawls sites to train Gemini and Vertex AI | |
| Meta-ExternalAgent | Meta | Crawls sites to train Meta AI models |
| Applebot-Extended | Apple | Crawls sites to train Apple Intelligence |
| FacebookBot | Meta | Crawls sites for Meta's language models |
| DeepSeekBot | DeepSeek | Crawls sites to train DeepSeek models |
| Bytespider | ByteDance | ByteDance crawler for LLM training |
| CCBot | Common Crawl | Common Crawl archive, used by many LLM projects |