Scrape-LE MCP server
Analyse robots.txt content and report whether a path may be crawled.
1 stars33 downloads/wk
Reviews
Write oneNobody has reviewed Scrape-LE yet.
If you have run it, two minutes of your experience saves the next person an afternoon.
Scrape-LE tools (1)
write = sends, deletes, buys or postsRead from the package source without running it. The installed server may list more.
analyze_robots_txtGiven the contents of a robots.txt file and a path, report whether the generic (User-agent: *) rules permit crawling it, along with the crawl delay, the disallowed patterns and any sitemaps. Takes the file contents directly and makes no network request of its own \u2014 fetch robots.txt with your own HTTP tool and pass what it returned.
Public scan report
scanner v0.1.9 · 2026-09-23 · same rubric, same numbers if you re-run it
- Code scan2 source files scanned25/25
- –Live reliabilityno gateway calls yet and no remote to proben/a
- –Tool poisoningtools not inspected (local package is not executed); not countedn/a
- Auth qualitylocal package, no credentials required12/15
- Maintenancelast push 1 days ago15/15
- Maintainer identityregistry namespace matches repository owner; GitHub account older than a year8/10
What the publisher says
From the Scrape-LE repository's README, as published. We do not edit it. Read it on GitHub
Scrape-LE: Zero Hassle Scrapeability Checks Load a URL in headless Chromium and see what will block your scraper — before you write it Anti-bot vendors, rate limits, robots.txt rules, login walls, console errors, screenshots
Useful? A star or rating is how other developers find it —
★ GitHub ·
★ Open VSX ·
★ Marketplace
What it does
Run Scrape-LE: Check URL Scrapeability (Ctrl+Alt+S / Cmd+Alt+S), enter a URL, and the page loads in a real headless Chromium. The report lands in the output channel: HTTP status, page title, load time, console errors, a full-page screenshot, and four detections. Works in VS Code and VS Code–based editors like Cursor and VSCodium (installable from Open VSX).
One-time setup: run Scrape-LE: Setup Browser to install Chromium (~130MB, into Playwright's browser cache).
Install
Use it from an AI agent
The same engine runs as an MCP server, so an agent can call it directly instead of you running a command.
analyze_robots_txt(content, path, maxResults?)Given robots.txt contents and a path, reports whether the generic (User-agent: *) rules permit crawling it, plus the crawl delay, disallowed patterns and any sitemaps.
Shortened. The full README is on GitHub.
Nothing above is checked by us. What we check is on the safety report.
Install directly
Runs npx -y scrape-le-mcp on your machine. Read the scan report first; the gateway never runs local packages.
claude mcp add scrape-le -- npx -y scrape-le-mcp
Scrape-LE: common questions
- Is Scrape-LE MCP server safe?
- Yes, by our scan: it is graded A (92/100). Read the Scrape-LE safety report
- How do I install Scrape-LE?
- It runs on your machine. Copy the Claude Code, Claude Desktop or Cursor config from the install section.
- Does Scrape-LE need an API key?
- Not as far as the registry entry and our scan can tell: no credentials are declared or required.
- Is Scrape-LE maintained?
- The last commit was 2 days ago (2026-09-22). The latest release is v2.2.5.
- What can I use instead of Scrape-LE?
- Servers from other publishers that do the same job: Contact Affordance MCP server and Business Contact Finder MCP server.
Alternatives to Scrape-LE
Same job from other publishers: the closest match first, then the best rated.
- Contact AffordanceCheck how to contact a business website, and whether that contact path actually works.not reviewedEstablishedA
- Business Contact FinderCheck how to contact a business website, and whether that contact path actually works.not reviewedEstablishedA
