X402 Data Gateway MCP server

SameDayDesk x402 and MPP Data Gateway

Optional bounded batch extraction

Paid GET /extract and GET /read (and MCP extract / read) return a typed record after settlement. Merchant HTTP 200 is delivery of that record, not proof that the source succeeded or that the page is complete. The JSON keeps requestedUrl, observed finalUrl (also copied to url for compatibility), source HTTP status, sourceOk, a nullable error, and a capture object that labels the no-JavaScript GET, body/excerpt limits, charset, and truncation. A 403/404 with block text is not an empty 200. Observational HTTP delivery evidence may classify schema conformance, source refusal, and truncation; that is not buyer usefulness or a sold guarantee. The extract text field is a bounded excerpt; use /read for longer Markdown. Neither route executes JavaScript, so missing discussion text is not proof of absence. Script data islands and comments are stripped; this is not a full-discussion reader. Production source connections validate and pin public IPv4 DNS answers, revalidate up to three redirects, and reject URL credentials, private addresses, IPv6 and compressed responses that ignore the requested identity encoding. Header/meta charset labels select a decoder; capture.charset names the decoder actually used. Unknown labels fall back to UTF-8 with an explicit fallback marker.

Batch rows retain requested source slots, including duplicates and failures. Each captured row's provenance.capture records the no-JavaScript method, actual byte allowance and 1,200-character text limit when text is requested. A known body or text cutoff is partial, not complete. This does not detect undisclosed missing content. Source refusal remains a failed row even when its error page has text. The in-process fetch/timeout test hooks have no HTTP, MCP or environment input binding and do not change settlement or authorization middleware.

EXTRACTBATCHENABLED=1 adds POST /extract/batch and MCP extractbatch at an introductory 0.01 USDC per admitted batch of 1–5 public HTTPS URLs. The default is off: 25 paid HTTP operations and 22 MCP tools; enabled: 26 and 23. This price is not a measured margin guarantee. Use one server process and persistent COMMERCEDATA_DIR; unknown settlement is quarantined, not automatically charged again. Bounded partial output does not promise every source or requested field.

LOCKFILEPINDELTAENABLED=1 adds POST /lockfile-pin-delta and MCP lockfilepindelta for two caller-supplied npm package-lock.json objects. The default is off. Production already has EXTRACTBATCHENABLED=1 (26 paid HTTP / 23 MCP tools). Enabling lockfile beside that envelope is 27 / 24. Default-off catalogs remain 25 / 22. Default price is $0.005 (LOCKFILEPINDELTAPRICE), matching live GET /extract. Margin is not proven. Initial live release accepts x402 only; existing MPP routes including extract_batch are unchanged. HTTP JSON only: no filesystem paths, arbitrary commands, or network fetch. Identical pins are informational, not engine failure. The signed deployment statement is not rewritten here. The lockfile MCP tool's challenge resource is https://agents.samedaydesk.com/lockfile-pin-delta, not mcp:// and not /extract/batch.

The extractbatch MCP tool projects the existing HTTP payment and durable replay handler. Its challenge resource is https://agents.samedaydesk.com/extract/batch, not mcp://: first call without payment, then reuse the returned HTTP resource and exact arguments with meta["x402/payment"]. MPP callers may use the unmodified www-authenticate challenge in result _meta["samedaydesk/http"].headers and send its credential as HTTP Authorization. That metadata also carries the HTTP status and receipt headers. Never translate an MCP-resource credential. Existing MCP tools retain their existing native MCP payment boundary.

Shortened. The full README is on GitHub.