Crawl Readiness MCP server

Crawl Readiness MCP Server

AI SEO tools inside Claude, Cursor, and other MCP-compatible clients.

Ask Claude "is my site AI-readable?" and it'll actually check — running a live audit against 50+ AI crawlers (ChatGPT, Claude, Perplexity, Google AI, and more). Then ask it to fix what's broken, and it can generate a proper llms.txt, robots.txt, or JSON-LD schema and write it straight into your project.

No more copy-paste between your editor and yet another SEO tool.

What it does

Eight tools your AI assistant can call:

Audit (no signup):

• checkaireadiness — Score a site 0–100 against 50+ AI crawlers, with a prioritized fix list
• validate_schema — Audit JSON-LD structured data on any URL (20+ types, per-type rules)
• validate_robots — Line-by-line robots.txt audit
• checkcontentparity — Compare what humans see vs what AI crawlers see

Generate (requires free API key):

• generatellmstxt — Auto-crawl a site and produce a properly formatted llms.txt
• generaterobotstxt — AI-crawler-aware robots.txt with preset policies
• generate_schema — Organization / WebSite / Article JSON-LD ready to paste

Monitor (read-only, requires free API key):

• getmonitortrend — See whether ChatGPT, Claude, Perplexity & Google AI mention your brand vs competitors, and how that's trending week over week (reads your LLM Monitor projects; doesn't trigger runs)

Install

For Claude Desktop

Open your config file:

• macOS: ~/Library/Application Support/Claude/claudedesktopconfig.json
• Windows: %APPDATA%\Claude\claudedesktopconfig.json

Add this to the mcpServers block:

{
  "mcpServers": {
    "crawl-readiness": {
      "command": "npx",
      "args": ["-y", "crawl-readiness-mcp"]
    }
  }
}

Restart Claude Desktop. You'll see "crawl-readiness" listed in the tools panel.

For Cursor

Open Cursor Settings → Features → Model Context Protocol → Add new global MCP server.

{
  "crawl-readiness": {
    "command": "npx",
    "args": ["-y", "crawl-readiness-mcp"]
  }
}

For Cline / Windsurf / Continue

Same shape — add crawl-readiness to your MCP servers list with npx -y crawl-readiness-mcp as the command.

Unlock the generator tools

The four audit tools work out of the box, no signup. The three generator tools need an API key — free at crawlreadiness.com/dashboard (50 checks/month on the free tier, no credit card).

Once you have a key, add it via env in the same config:

{
  "mcpServers": {
    "crawl-readiness": {
      "command": "npx",
      "args": ["-y", "crawl-readiness-mcp"],
      "env": {
        "CRAWL_READINESS_API_KEY": "cr_live_xxxxxxxxxxxx"
      }
    }
  }
}

Restart the client. All eight tools are now callable.

Example prompts

Once installed, try any of these in your AI client:

• "Is my portfolio site AI-readable? URL: yourdomain.com"
• "Check if ChatGPT can see acme-inc.com"
• "Validate the JSON-LD on stripe.com — is anything missing?"
• "Compare what humans see vs what AI crawlers see on producthunt.com"
• "Generate an llms.txt for my site and save it to /public/llms.txt"
• "My site scored 68/100 — walk me through the fixes"

Your AI can chain tools automatically — audit, identify issues, fix them, and write files to your project without you copy-pasting anything.

Configuration

What tools receive from the server

Each tool returns the full JSON response from the Crawl Readiness API — the same data the web dashboard renders. Your AI has access to:

Shortened. The full README is on GitHub.