WebScraping.AI MCP server

WebScraping.AI MCP Server

Prefer zero setup? Use the hosted remote MCP server:

add https://mcp.webscraping.ai/mcp to your MCP client and sign in with your

WebScraping.AI account — OAuth handles auth, no API key or local install

needed. This repo is the open-source stdio version for self-hosting and

customization.

A Model Context Protocol (MCP) server implementation that integrates with WebScraping.AI for web data extraction capabilities — Chromium JavaScript rendering, rotating datacenter/residential/stealth proxies, and AI-powered question answering and structured field extraction on any page.

Sign up to get an API key — the free trial includes 2,000 credits, no credit card required. See the API documentation for the full parameter reference.

Features

• Question answering about web page content
• Structured data extraction from web pages
• HTML content retrieval with JavaScript rendering
• Plain text extraction from web pages
• CSS selector-based content extraction
• Multiple proxy types (datacenter, residential, stealth) with country selection
• JavaScript rendering using headless Chrome/Chromium
• Concurrent request management with rate limiting
• Custom JavaScript execution on target pages
• Device emulation (desktop, mobile, tablet)
• Account usage monitoring
• Content sandboxing option - Wraps scraped content with security boundaries to help protect against prompt injection

Installation

Running with npx

env WEBSCRAPING_AI_API_KEY=your_api_key npx -y webscraping-ai-mcp

Manual Installation

# Clone the repository
git clone https://github.com/webscraping-ai/webscraping-ai-mcp-server.git
cd webscraping-ai-mcp-server

# Install dependencies
npm install

# Run
npm start

Configuring in Cursor

Note: Requires Cursor version 0.45.6+

The WebScraping.AI MCP server can be configured in two ways in Cursor:

Create a .cursor/mcp.json file in your project directory:

1. Project-specific Configuration (recommended for team projects):

{
     "servers": {
       "webscraping-ai": {
         "type": "command",
         "command": "npx -y webscraping-ai-mcp",
         "env": {
           "WEBSCRAPING_AI_API_KEY": "your-api-key",
           "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5",
           "WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING": "true"
         }
       }
     }
   }

Create a ~/.cursor/mcp.json file in your home directory with the same configuration format as above.

1. Global Configuration (for personal use across all projects):

If you are using Windows and are running into issues, try using cmd /c "set WEBSCRAPINGAIAPI_KEY=your-api-key && npx -y webscraping-ai-mcp" as the command.

This configuration will make the WebScraping.AI tools available to Cursor's AI agent automatically when relevant for web scraping tasks.

Running on Claude Desktop

Add this to your claudedesktopconfig.json:

{
  "mcpServers": {
    "mcp-server-webscraping-ai": {
      "command": "npx",
      "args": ["-y", "webscraping-ai-mcp"],
      "env": {
        "WEBSCRAPING_AI_API_KEY": "YOUR_API_KEY_HERE",
        "WEBSCRAPING_AI_CONCURRENCY_LIMIT": "5",
        "WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING": "true"
      }
    }
  }
}

Configuration

Environment Variables

Required

• WEBSCRAPINGAIAPI_KEY: Your WebScraping.AI API key
• Required for all operations
• Get your API key from WebScraping.AI

Shortened. The full README is on GitHub.