Agent Readiness
What the agent-readiness scan actually verifies, category by category. Points shown are maximums; a partial earns half, and checks that can't be measured are excluded with the score renormalized (see the overview for scoring and grades).
Access: can an agent fetch anything at all? (25 pts)
robots.txt AI-bot rules (8) — Parses your robots.txt for the major AI crawlers and fetchers. Blocking training crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot) is a defensible policy choice and only costs partial credit — but blocking answer-time fetchers (ChatGPT-User, OAI-SearchBot, Claude-User, Perplexity-User) fails the check, because it kills live retrieval even when a user directly asks about your product. Pass: don't disallow the answer-time fetchers. No robots.txt at all means nothing is blocked, which passes.
Bot-access matrix (8) — Fetches your homepage, docs, and llms.txt with a normal browser user-agent (the control) and again with AI-bot user-agents, looking for 401/403/429/challenge responses that only hit the bots. Pass: no AI-bot UA gets blocked where the browser succeeds. Caveat we state in every report: this is UA-level only — a pass doesn't prove real bot IP ranges get through WAF verification, but a fail is a definite problem. Fix: allowlist AI agent user-agents (or the vendors' published IP ranges) in your WAF/bot-protection layer.
Soft-404 hygiene (4) — Asks for .txt/.md/.json paths (including one guaranteed-nonexistent canary) and flags any that return 200 with an HTML app shell. A SPA router that serves the app for every URL means agents can never trust a 200 from your site. Pass: unknown routes return real 404s, and file paths serve real files with text content types.
Docs readable without JS (5) — Extracts the visible text from your docs page's raw HTML, the way an agent without a browser sees it. An empty JS shell fails; thin partial server-rendering earns partial. Pass: server-render (SSR/SSG) so the full content is in the initial HTML response.
Discovery: can it find your machine entry points? (20 pts)
llms.txt present + spec-valid (8) — The same structural checks as our validator: H1 title, blockquote summary, H2 sections, markdown links. Full pass = all structural checks pass; warnings = partial; missing or failing = zero. Fix: publish /llms.txt — the generator produces a spec-compliant one.
llms-full.txt present (3) — Probes for the companion full-content file so agents can load everything in one fetch. A file larger than ~5 MB earns only partial — that's beyond any agent's context window, so "full content in one fetch" doesn't actually work at that size.
sitemap.xml present + parses (3) — Fetches the sitemap (preferring the one your robots.txt declares) and checks it's valid sitemap XML. Pass: valid XML, referenced from robots.txt with a Sitemap: line.
Docs discoverable from root (4) — Can a docs surface be reached from your homepage: via llms.txt links, conventional paths (/docs, /documentation), or homepage navigation? If an agent can't find your docs from your front door, nothing downstream matters.
Other agent files (2) — Presence probes for site-level AGENTS.md, skill.md, and .well-known agent files. Informational weight only — these are usually repo-scoped conventions; site-level presence is a bonus signal.
Comprehension: can it read what it fetches? (25 pts)
Markdown availability (6) — Tries .md mirrors of your docs pages, Accept: text/markdown content negotiation, and whether docs are already served as clean markdown. Agents parse markdown far more reliably (and cheaply) than themed HTML. Having only llms-full.txt earns partial.
Quickstart exists & is shallow (6) — Looks for a quickstart/getting-started page within two hops of your llms.txt or docs root. One that's only findable by guessing URLs earns partial — an agent may never find it. Fix: link the quickstart from llms.txt's first section and your docs nav.
Structure quality (5) — On the quickstart/docs page: exactly one h1, real h2 sections, and code samples in actual <pre>/<code> blocks rather than mangled into prose.
Token budget (4) — Estimates token counts (chars ÷ 4): llms.txt should stay under ~4,000 tokens and a quickstart page under ~25,000, or agents truncate or skip them. Fix: split oversized surfaces; move detail into linked pages.
Link liveness (4) — Samples links from your llms.txt and quickstart and follows them. Dead links fail agents that follow them literally — which is all of them.
Integration: can it act on your product? (30 pts)
(Skipped entirely — and the score renormalized — for sites with nothing to integrate.)
OpenAPI spec present + parses (7) — Probes conventional paths (/openapi.json, /openapi.yaml, /swagger.json, /.well-known/) and spec files linked from docs. Full credit requires the spec to parse and describe its auth scheme — an agent that can see your endpoints but not how to authenticate is still stuck.
Package-hallucination check (8) — Extracts install commands (npm install …, pip install …) from your quickstart/docs and verifies each name resolves on the registry. Also probes the obvious names an agent would guess (<product>, <product>-sdk, @<product>/sdk) — an unclaimed obvious name means a guessing agent gets a 404, or worse, a future squat. Fix: make sure documented names resolve; claim obvious names as stubs.
Quickstart 404 scan (6) — Follows every URL your quickstart references (links and endpoints in code samples). 2xx and auth-walls (401/403) are fine; 404s are exactly where agents die.
Programmatic auth path (5) — Do your docs contain a copyable auth example (Authorization: Bearer …, x-api-key header) an agent can put in a request? Key mentions without a literal example earn partial; a purely dashboard-clicking flow with no documented key usage fails.
MCP presence (4) — Probes /.well-known/mcp.json and looks for a documented MCP endpoint. An emerging surface with deliberately low weight — but increasingly how agents integrate without writing HTTP glue.
Run the free scan to see where you stand, compare against the Index, or have a real agent attempt your onboarding end to end.