Index monitoring
Find out before Google drops your pages.
One deploy can add a noindex, block a crawler, or ship an empty page. IndexMon checks every page the way Googlebot and AI crawlers see it — and emails you the fix.
No signup · 6 layers · 9 crawlers · 15 seconds
- 14:02:11↑deploy a81f3c → production
- 14:02:19✕/pricing X-Robots-Tag: noindex added
- 14:02:19✕/blog/* 412 → 9 words in HTML (JS-only)
- 14:02:20@alert emailed with the fix → next.config.js headers()
- 14:31:47↑deploy b02d9e → production
- 14:31:55✓/pricing indexable again
Free deep check
See what Google sees. Right now.
Paste any URL. We fetch it like Googlebot, ask nine crawlers, render the JavaScript, check the domain and certificate, then write a fix plan for your stack.
The problem
One line of config. Weeks of lost traffic.
A site owner left Disallow: / live for six days. Organic traffic fell 20% — and stayed down after the fix while Google re-crawled. Search Console tells you days later. By then the damage is done.
Source: Moz Q&A, “Recovering from robots.txt error”
User-agent: * Disallow: / # copied from staging Sitemap: https://acme.dev/sitemap.xml
Six layers deep
Every page, checked the way crawlers actually work.
Each layer gets its own score. A failure in any of them is enough to keep a page out of Google.
Access
Can a crawler reach the page at all?
HTTP status · redirect chain · TTFB · HTTPS / HSTS · compression · page weight
Crawlability
Who is allowed in — by robots.txt and by your firewall.
robots.txt per crawler · Googlebot · Bingbot · GPTBot · ClaudeBot · PerplexityBot · WAF / CDN blocks
Indexability
Is Google told to index this URL, or another one?
meta robots · X-Robots-Tag · canonical · sitemap membership · hreflang
Content
Is there enough on the page to be worth indexing?
title · description · H1 · word count · structured data · alt text · internal links
JavaScript rendering
What exists only after JS runs — invisible to AI crawlers.
HTML vs rendered text · JS-injected links · JS-added noindex · console errors
Domain & SSL
The renewals that take whole sites offline.
certificate expiry (CT logs) · domain expiry (RDAP) · nameservers · DNS
Built with Lovable, Bolt or v0?
Your page might be empty to crawlers.
Many AI-built sites ship a blank HTML shell and draw the page with JavaScript. Google indexes it late, if at all. GPTBot and ClaudeBot don't run JavaScript — they see nothing. We render the page in a real browser and measure the gap.
<body>
<div id="root"></div>
<script type="module"
src="/assets/index-Bx12.js">
</script>
</body>fix → prerender or server-render the page so the HTML already contains the text.
Search + AI crawlers
ChatGPT can't cite a page it can't read.
We test nine crawlers against your robots.txt and your firewall. Blocking AI training bots can be a choice. Blocking ChatGPT search or Perplexity by accident costs you traffic.
| Crawler | Type | robots.txt | Firewall |
|---|---|---|---|
| Googlebot | Search | allowed | 200 |
| Bingbot | Search | allowed | 200 |
| Applebot | Search | allowed | 200 |
| OAI-SearchBot | AI search | allowed | 200 |
| PerplexityBot | AI search | allowed | 403 |
| GPTBot | AI | blocked | 403 |
| ClaudeBot | AI | allowed | 403 |
| Google-Extended | AI | blocked | 200 |
| CCBot | AI | blocked | 200 |
example · PerplexityBot and ClaudeBot allowed by robots.txt but blocked by the firewall
Alerts that fix things
An email with the fix, not a dashboard you forget.
IndexMon checks daily and right after every deploy. When a page flips from indexable to not, you get one email: what broke, where, and the exact change for your stack — Next.js, WordPress, Shopify, Webflow, Lovable and more.
Deploy a81f3c added a X-Robots-Tag: noindex header to /pricing. Google will drop the page on its next crawl.
- { key: "X-Robots-Tag", value: "noindex" }
Detected stack: Next.js on Vercel · open full report
For agencies
Catch it before your client does.
Watch every client site from one place. Send white-label reports and a live link your client can open — with your logo, not ours.
- Up to 50 sites, 5,000 pages
- White-label PDF + shareable client report
- One alert inbox for all clients
- Deploy hooks per site
example client list
Pricing
Start free. Pay when it saves you.
Early access is open — monitoring is free while we roll out paid plans.
Pro
$19per month
- ✓5 sites · 500 pages
- ✓Daily checks + deploy hook
- ✓AI fix plans for your stack
- ✓Search Console index status
- ✓Change history
Agency
$79per month
- ✓50 sites · 5,000 pages
- ✓Everything in Pro
- ✓White-label client reports
- ✓Shareable client links
- ✓Team seats
FAQ
Questions
Is the deep check really free?+
Yes. Run it on any URL, no signup. Monitoring (daily + after every deploy) is free during early access.
Does it tell me if a page is in Google's index?+
It tells you whether Google can index it and why not. Live index status from Search Console is part of monitoring.
How is this different from Search Console?+
Search Console reports problems days later, after traffic drops. IndexMon checks within minutes of a deploy and tells you the exact fix.
Why check AI crawlers?+
ChatGPT, Claude and Perplexity can only cite pages their crawlers can read. Many sites block them by accident — in robots.txt or at the firewall.
My site is built with Lovable / Bolt / v0. Is that a problem?+
Often. These apps ship an empty HTML shell and build the page with JavaScript. We measure exactly how much content crawlers miss.
Your next deploy is coming.
Know what it does to Google before your traffic does.