Crawler Check.
Check whether AI crawlers can read your homepage, and whether you publish a sitemap and llms.txt.
See who can read your site.
This is the technical slice of search and AI visibility. For the full picture, order the SEO + GEO Audit.
Frequently asked questions.
01What is robots.txt?
It is a public file that tells crawlers which paths they may fetch. No file means they are allowed by default. A rule for one bot does not change the rules for the others.
02Should I block AI crawlers?
Blocking GPTBot or Google-Extended keeps your pages out of those training and answer products. Leaving them allowed makes it possible to be cited. Either choice is valid. This tool only shows which one you made.
03What is llms.txt?
It is an optional text file at /llms.txt that points models at the pages you want them to use. A sitemap still matters more for classic search.