# ClaudeBot Tracker

Separate Anthropic crawler policy from verified request evidence and downstream Claude visibility outcomes.

Canonical URL: https://truborankai.com/track-claudebot

![AI crawler requests moving through access verification, server logs, traffic timelines, and evidence checks](/assets/img/blog/ai-crawler-monitoring-cluster.webp)

## Quick Answer

To track ClaudeBot correctly, define whether you mean ClaudeBot, Claude-SearchBot, or Claude-User; inspect the deployed robots.txt rule for the named identity; collect server, CDN, edge, or WAF requests; match the exact User-Agent; verify against Anthropic guidance and the current source IP list where available; record requested URLs, status codes, first and last seen times, and uncertainty; and never present a crawler hit as proof that Claude trained on, cited, or recommended the page.

## AI Summary

This provider-specific tracker covers Anthropic crawler families, access intent, verification, traffic evidence, and safe interpretation in one operational workspace.

## Main Explanation

Anthropic documents separate crawler identities. ClaudeBot is associated with model development, Claude-SearchBot supports search-related discovery, and Claude-User is associated with user-directed retrieval. Build filters and policy notes around those published names rather than grouping every request containing “Claude” into one chart.

Test the deployed policy for representative paths. A global rule can differ from a documentation, media, parameter, or private directory rule. Anthropic documents robots support and a Crawl-delay behavior, but edge security, rate limits, authentication, and origin health can still determine the actual response.

For observed traffic, retain the raw request and mark verification status. Compare the source address with Anthropic’s current guidance when feasible and investigate forwarded-address handling behind a proxy. A verified request plus a successful response is meaningful crawl evidence, but it still stops short of proving later use.

Review coverage after identity verification. Compare requested Claude crawler paths with the canonical sitemap and priority content inventory. A page with no observed request may be new, weakly linked, outside the retention window, cached at another layer, or simply not selected for crawling; report these possibilities instead of calling absence a block.

Keep a small evidence record for every important observation: canonical URL, requested path, timestamp and timezone, status code, response bytes, User-Agent, verification state, and the edge or origin source. Aggregate by provider purpose only after preserving raw evidence. Sampling, caching, proxying, log retention, and privacy controls can change totals, so document those boundaries beside the chart.

Separate crawler families by purpose. OpenAI documents GPTBot, OAI-SearchBot, and ChatGPT-User separately; Anthropic distinguishes ClaudeBot, Claude-SearchBot, and Claude-User; Perplexity distinguishes PerplexityBot and Perplexity-User. A policy intended for training collection may not be the policy intended for search discovery or a user-triggered fetch.

Use the result as a diagnosis, not a visibility score. A verified request proves a resource was requested at a particular time. It does not prove indexing, model training, answer inclusion, citation, recommendation, referral traffic, or conversion. Measure those outcomes with their own named reports and dates.

## Practical Steps

1. Name the Claude crawler identity and purpose.
2. Inspect robots.txt for representative public paths.
3. Confirm edge and origin access behavior.
4. Collect infrastructure requests.
5. Match the exact User-Agent.
6. Apply current verification evidence.
7. Report paths, statuses, times, and uncertainty.
8. Retest after an intentional policy or server change.

## FAQ

### Are ClaudeBot and Claude-SearchBot the same?

No. Anthropic documents them as separate identities with different purposes.

### Does robots.txt prove ClaudeBot visited?

No. It describes policy; a visit requires server-side request evidence.

### Does a ClaudeBot request prove a Claude citation?

No. Crawling and answer citation are separate outcomes.

## Editorial Methodology

Reviewed on 2026-08-26 against official OpenAI, Anthropic, Perplexity, and Cloudflare crawler documentation. The workflow separates published policy, request identity, observed traffic, search use, citations, referrals, and conversions and makes no guaranteed visibility claim.

## Sources

- [OpenAI: crawler purposes, controls, and published IP ranges](https://developers.openai.com/api/docs/bots)
- [OpenAI: publisher controls and ChatGPT referral attribution](https://help.openai.com/en/articles/12627856-publishers-and-developers-faq)
- [Anthropic: ClaudeBot, Claude-SearchBot, and Claude-User](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)
- [Perplexity: crawler identities and verification guidance](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)
- [Cloudflare: AI crawler directory and categories](https://developers.cloudflare.com/ai-crawl-control/reference/bots/)

## Related Internal Links

- [How to Track ClaudeBot Traffic](/blog/how-to-track-claudebot)
- [AI Bot Traffic Checker](/ai-bot-traffic-checker)
- [AI Bot Access Checker](/ai-bot-access-checker)
- [AI Crawler Tracking Guide](/blog/how-to-track-ai-crawlers)
- [GPTBot Checker](/track-gptbot)
- [PerplexityBot Tracker](/track-perplexitybot)