🤖 AI BOT VISIBILITY

Monitor every AI crawler hitting your site.

GPTBot, ClaudeBot, PerplexityBot, Amazonbot, OAI-SearchBot, Bytespider — they're indexing your content right now. Whoever ranks in their answers wins the next decade of search. But you can't optimize for what you can't measure.

SysWP Radar tracks every AI crawler request in real time: which bot, which pages, how often, from where. Server-side capture — no JavaScript required (because AI crawlers don't run JS).

Start free · 7-day Pro trial See AI bot dashboard

Why your current tools can't see AI crawlers

Google Analytics, Plausible, Fathom, and every other JS-pixel analytics tool fire from inside the visitor's browser. AI crawlers fetch content over plain HTTP with no browser. They send a User-Agent string identifying themselves, they hit your URLs, they download HTML — and they leave without ever executing your <script> tags.

Result: your "traffic report" shows 0 visits from OpenAI, Anthropic, and Perplexity, while their bots may actually be the #1 source of indexing on your site.

The AI crawlers Radar tracks

Each one has a different role in the AI economy:

  • GPTBot — OpenAI's training crawler + ChatGPT search citations. UA: Mozilla/5.0 ... GPTBot/1.x. Cited content appears in ChatGPT answers (Search GPT).
  • ClaudeBot — Anthropic's crawler for Claude training and Claude.ai citations. UA: ClaudeBot/1.0; +claudebot@anthropic.com.
  • PerplexityBot — Perplexity's real-time answer engine crawler. UA: Mozilla/5.0 (compatible; PerplexityBot/1.0). Cited pages appear in Perplexity's source pills.
  • Amazonbot — Amazon's Alexa + Rufus shopping AI. UA: Mozilla/5.0 (Linux; Amazonbot/0.1). Important for e-commerce visibility.
  • OAI-SearchBot — OpenAI's standalone search crawler (separate from GPTBot). UA: OAI-SearchBot/1.0.
  • Bytespider — ByteDance / TikTok AI training. UA includes Bytespider.
  • YouBot — You.com search engine.
  • cohere-ai — Cohere AI training crawler.
  • Anthropic-AI — secondary Anthropic crawler (separate from ClaudeBot).
  • FacebookExternalHit — Meta's Llama training + Facebook link previews.
  • Google-Extended — Google's separate signal for Bard / Gemini training opt-out.

What you can do with AI crawler data

  • SEO for AI answers (GEO). See which pages each crawler revisits. If GPTBot returns repeatedly to /blog/wordpress-attacks-q1/, that page is being weighted as a strong answer source — double down on similar content.
  • Bandwidth optimization. AI crawlers can consume 20-50% of server resources on content-heavy sites. Now you know which.
  • Content gap detection. If a crawler hits 404s a lot, that's content people are asking AI assistants about that you don't have.
  • Opt-out enforcement verification. If you set robots.txt rules to block specific crawlers, verify they're actually respecting them. Many do; some don't.

How to set it up

SysWP Radar's classifier auto-detects all known AI crawlers without any configuration. Install the WordPress plugin (or paste one line of JavaScript for any other platform) and the AI bot panel populates within minutes.

  1. Create free account
  2. Add your site → grab the 12-character Site ID
  3. WordPress: install the SysRadar plugin + paste Site ID in Settings → SysRadar
  4. Other platforms: paste the script tag before </head>

Within 10 seconds of the next AI crawler visit, you'll see it in Dashboard → Top AI bots: name, hit count, top pages crawled, country of origin (most AI crawlers run from US-East AWS).

FAQ

Can I block GPTBot / ClaudeBot if I want?
Radar detects them; it doesn't block. To block, add Disallow rules to robots.txt (most major crawlers honor it) or use a WAF rule against the User-Agent. Or pair Radar with SysWP Shield for one-click rule deployment.
How accurate is the AI crawler classification?
We match against published User-Agent strings + verify reverse-DNS where the crawler publishes its IP ranges (Google, OpenAI). False positives are rare; false negatives can happen if a crawler ships without announcing in advance — we update the classifier when new AI products launch.
Does this work without WordPress?
Yes. The JavaScript pixel works on any platform. The WordPress plugin adds extra server-side capture for the WordPress-specific blind spots (wp-json, xmlrpc, etc.) — but the core AI crawler detection works everywhere.
Does it work in real time?
Yes. Events appear in the dashboard within 5-10 seconds of the crawler hit. The alerts engine can also fire when a specific crawler appears for the first time (useful for tracking new AI launches as they happen).

See which AI is reading your content.

Free forever plan. Most sites see the first GPTBot or ClaudeBot hit within an hour of install.

Create free account →

First 100 paying customers lock in 50% lifetime discount.

Related topics