AI assistants
AI Assistant Bots
These bots fetch your pages when a user explicitly asks an AI assistant to look up information from the web.
- bots profiled
- 9
- respect robots.txt
- 1
- publish IP verification
- 2
Bots in this category
- ChatGPT Agent OpenAI No robots.txt commitment
- ChatGPT-User OpenAI robots.txt not documented
- Devin Devin AI No robots.txt commitment
- DuckAssistbot DuckDuckGo Respects robots.txt
- Meta-ExternalFetcher Meta No robots.txt commitment
- MistralAI-User Mistral AI No robots.txt commitment
- Amazon Buy For Me Amazon No activity data No robots.txt commitment
- Perplexity-User Perplexity AI No activity data No robots.txt commitment
- TwinAgent Twin No activity data No robots.txt commitment
Crawl activity, 25 Oct - 21 Nov 2026 · Activity trend: Cloudflare Radar
About this category
AI assistant bots are user-initiated - they only crawl your site when someone asks an AI chatbot or assistant to browse or reference web content. Each request represents a real user seeking your information through an AI intermediary.
Block every AI assistants in one go
Rules for the 3 bots in this category that document a robots.txt token. Bots without a token can't be targeted this way.
# AI assistants
User-agent: ChatGPT-User
User-agent: DuckAssistBot
User-agent: Perplexity-User
Disallow: / Waitlist open
Know which of these bots visit you
Can AI See It shows which bots from this catalogue hit your site, which are fake and what each platform sends back.