What your origin serves a declared agent, and what it publishes for one.

Ten well-known documents — the three policy documents, then MCP and plugin manifests, the long-form llms.txt and an OpenAPI description at four paths — requested under two client identities that differ only in what they say they are, after your robots.txt is read and obeyed. It reports what came back. It does not score anything.

Only /robots.txt, /llms.txt, /sitemap.xml and the seven well-known documents above are ever requested, and only after your own robots.txt is read and obeyed — twice, once per scope, because consent is not taken from a cache. No page content is fetched, and no request is ever made under another crawler’s name. What the crawler is and how to block it is on the crawler page.