Check a domain

Paste any public domain to find contradictions between its robots.txt and sitemap files.

Enter a domain to check

Robotsdiff fetches robots.txt and every sitemap it can find, then reports URLs that appear in a sitemap and match a Disallow rule — pages you explicitly asked Google to index and explicitly told it not to read.

What this tool does not do: It does not judge whether a page should be indexed, fetch the pages themselves, or estimate ranking impact. It compares two declarations and names where they disagree. Results are based on the files as served right now — cached or stale versions may differ.

Cloudflare-hosted domains: A Worker cannot reach a host that is itself behind Cloudflare. If your domain uses Cloudflare, this tool may report it as unreachable even though the site is up. This is a platform limitation, not a problem with your server.