Crawlgraph MCP server

mcp-crawlgraph

CrawlGraph MCP — backlink intelligence on the public Common Crawl webgraph

Part of Pipeworx — an MCP gateway connecting AI agents to 1576+ live data sources.

Tools

Quick Start

Add to your MCP client (Claude Desktop, Cursor, Windsurf, etc.):

{
  "mcpServers": {
    "crawlgraph": {
      "url": "https://gateway.pipeworx.io/crawlgraph/mcp"
    }
  }
}

What this endpoint actually serves

tools/list at https://gateway.pipeworx.io/crawlgraph/mcp returns the tools in the table above plus the shared Pipeworx meta-tools — askpipeworx, discovertools, search_within, remember/recall and the rest of the gateway-wide set. So the tool count you see is larger than this table: a single-pack endpoint currently lists roughly 30 shared tools alongside the pack's own. The connection's initialize response states its exact scope, and is the authoritative answer for a given day.

This is deliberate, not multiplexing by accident. The meta-tools are what let a scoped connection answer a question this pack does not cover — via ask_pipeworx, which routes across the whole catalog — without you adding a second MCP server. There is currently no way to mount a pack endpoint without them; if the extra schemas cost you more context than the routing is worth, connect to the full gateway once rather than to several pack endpoints.

Or connect to the full Pipeworx gateway to get every pack's tools listed directly, instead of just this one's:

{
  "mcpServers": {
    "pipeworx": {
      "url": "https://gateway.pipeworx.io/mcp"
    }
  }
}

Both URLs reach the same gateway and the same 1576+ data sources. The only difference is which pack's tools are listed directly; ask_pipeworx reaches all of them from either one.

No MCP client? Call it over HTTP

Shortened. The full README is on GitHub.