Skip to main content

AI Bot Analytics

The AI Bot Analytics module shows how AI platform bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and others) crawl your site. Before AI can cite content, it has to visit it first — this module shows whether that's happening.

What it's for​

  • Records AI bot visits per URL: which bot, when, how often.
  • Shows the most crawled pages and the most active bots.
  • Computes the crawl → citation correlation: which crawled pages actually become cited sources in AI answers.
  • Flags problems: if bots don't visit your key pages, that content has no chance of appearing in answers.

The monthly limit of imported visits depends on your package — see Limits.

How to start​

The module requires a site behind Cloudflare — that's where we pull bot traffic logs from. You need your Cloudflare Zone ID and an API token, entered in the company profile. Import runs automatically every hour.

Step by step​

  1. In the Cloudflare dashboard, copy your domain's Zone ID and create an API Token with log/analytics read permission.
  2. Open the company profile and paste both in the Cloudflare section. Save.
  3. Open AI Bot Analytics — after the first import you'll see the KPIs: bot visit count, unique pages, the most active bot and the bot → citation correlation.
  4. Review the path table: which pages bots visit and which they skip. Compare it against the pages you care about (platform-published content, product pages).
  5. Check the cited pages panel: how many pages cited in AI were also crawled — confirmation that crawling translates into presence in answers.
Bots not visiting your pages?

Check the Crawler monitor — it verifies robots.txt, HTTP availability, structured data and your sitemap for AI bots. A GPTBot blocked in robots.txt is the most common cause.

FAQ​

Does the module work without Cloudflare? No — Cloudflare logs are the data source. If your site sits behind a different CDN/server, consider moving DNS through Cloudflare (the free plan is enough for bot data).

What does exceeding the visit limit mean? Import and history keep working — you'll see a banner about the overage. The limit is a package dimension, not a hard data cutoff.

Which bots do you track? Among others: GPTBot, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended, CCBot, Bytespider — we extend the list as new relevant AI crawlers appear.

Does high bot traffic mean high visibility? Not necessarily — crawling is a necessary condition, not a sufficient one. That's why we compute the crawl → citation correlation instead of raw visit volume.