QueryRev AI Readiness

Checks how a site's public files treat AI crawlers: which of 11 AI crawlers robots.txt lets in, whether the llms.txt is a real llms.txt, and what the home page says without running scripts. Every result is a finding with its evidence. There are no scores, no model calls and no telemetry.

npx @queryrev/ai-readiness check www.example.com

Requires Node 22.19 or later. One runtime dependency (undici).

Install

npx @queryrev/ai-readiness <command>      # no install
npm install --global @queryrev/ai-readiness
ai-readiness <command>

Commands

Options: --json, --fail-on fail|warn, --allow-private-network, --no-color, --help, --version. Options go after the command.

check

$ ai-readiness check www.example.com
AI readiness check: https://www.example.com

Crawler access (robots.txt, whole site)
  Crawler             Purpose   Access
  GPTBot              training  allowed
  OAI-SearchBot       search    allowed
  ...

Findings
  PASS  robots-txt                  robots.txt found (77 bytes).
  PASS  sitemap-in-robots           robots.txt lists a sitemap.
        | https://www.example.com/sitemap.xml
  PASS  ai-search-crawler-access    All 5 AI search and user-fetch crawlers on the list can read the home page.
  INFO  ai-training-crawler-access  No AI training crawler on the list is blocked. Blocking them is the site's choice.
  WARN  organization-same-as        The Organization has no sameAs link to a profile on another site.
  ...

13 pass, 1 warn, 0 fail, 2 info

check asks for four files, in this order, and only where robots.txt lets this tool in: /robots.txt, the home page, /llms.txt and the sitemap. If robots.txt cannot be read, nothing else is requested and the exit code is 3.

Training crawlers are reported but never counted as a failure: blocking them is the site's choice. A crawler for AI search or for fetching a page when a user asks is a fail when robots.txt keeps it from the home page.

robots test

$ ai-readiness robots test robots.txt --agent OAI-SearchBot --path /account/settings
  FAIL  robots-access  OAI-SearchBot may not fetch /account/settings: the "*" group (line 9), decided by line 10 (Disallow: /account/).
        | line 10: Disallow: /account/

With --agent the exit code answers the question: 0 when the crawler may fetch the path, 1 when it may not. Any token can be tested; the tool suggests only the crawlers it covers. Without --agent it tests every covered crawler.

Matching follows RFC 9309: the group that names the token applies, else the * group; the longest matching rule wins and Allow wins a tie.

llms validate

$ ai-readiness llms validate https://www.example.com --links
  PASS  llms-txt-found     The file is an llms.txt.
  PASS  llms-txt-h1        The file has one H1, as llmstxt.org asks.
  PASS  llms-txt-summary   The file has a summary blockquote.
  PASS  ll

... [View full README on GitHub](https://github.com/queryrevai/ai-readiness#readme)