Back to Marketplace

Web Scraping Tools

Web scraping and content extraction tools

Get Started NowDocumentation

Overview

Web Scraping Tools is a production-ready Model Context Protocol (MCP) server built to give AI agents reliable, structured access to web scraping tools capabilities.

It acts as a standardized bridge between large language models and real-time data. Instead of relying on static training knowledge, models can retrieve live results, web content, and intelligence through a controlled MCP interface.

By integrating this MCP server, developers enable models such as Claude, GPT-4, Gemini, and open-source LLMs to:

• Execute structured Web Scraping Tools queries • Access live data in real time • Retrieve specialized information programmatically • Ground responses in verifiable external sources

This architecture significantly improves factual accuracy, reduces hallucination risk, and expands what AI systems can accomplish in research, automation, monitoring, and decision-support workflows.

Web Scraping Tools is designed for teams building production AI agents that require dependable, real-time access to the web scraping within a standardized MCP ecosystem.

Highlights

Protocol
MCP v1.0
Security
OAuth 2.0
Access
Real-time
Tools
1 Tools

Standardized bridge for real-time model context.

Installation

Connect this server to your local or remote agent environment.

mcp_config.json
{
  "mcpServers": {
    "mcp360": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://connect.mcp360.ai/v1/web-scraping/mcp?token=YOUR_API_KEY"
      ]
    }
  }
}
Replace YOUR_API_KEY with your actual key from the dashboard.

Skills & Capabilities

What this MCP server can do for your AI agents.

CAPABILITIES
--- name: web-scraping description: Extract and convert webpage content to clean markdown or JSON format, scrape HTML elements, retrieve page text, and parse structured data from any URL. Use when users need to extract article content, scrape website data, convert HTML to markdown, analyze web pages, or automate content collection from multiple sites. metadata: author: MCP360 version: "1.0.0" tags: - Web Scraping - Data Extraction - HTML Parsing - Content Extraction - Markdown Conversion - Web Automation - Data Collection - API Integration keywords: - scraping - web - content - extraction - html - markdown - page - crawl - data - text - parse - fetch - url - api creditsPerUse: 1 isPremium: false --- # Web Scraping Tools Professional web scraping API for content extraction, HTML parsing, and data collection. Extract clean text from web pages, convert HTML to markdown, scrape structured data, and automate content retrieval from any public website with intelligent parsing and format conversion. ## Core Capabilities - **Page Scraping**: Extract complete webpage content including text, links, images, and metadata, and returns cleaned HTML, markdown-formatted content, raw text, meta tags, page title, structured data, and all embedded media URLs for content analysis and data mining ## API Access ### MCP360 Connect Endpoints - **Base URL**: `https://connect.mcp360.ai/api/v1` - **Service Path**: `web-scraping` - **Authentication**: Bearer token (API key required) ### Service Metadata Endpoint ```bash curl "https://connect.mcp360.ai/api/v1/web-scraping" \ -H "Authorization: Bearer YOUR_API_KEY" ``` ### Tool Execution Endpoint ```bash curl -X POST "https://connect.mcp360.ai/api/v1/web-scraping/{tool_name}" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{...parameters...}' ``` ## Available Tools ### Scrape Page Extract content from any webpage with automatic HTML parsing and markdown conversion. **Tool Name:** `scrape_page` **Credits Required:** 1 **Parameters:** - `url` (string, required): Full webpage URL to scrape like "https://example.com/article" or "https://blog.example.com/post/123" - `format` (string, optional): Output format - "markdown" for clean text, "html" for raw HTML, "text" for plain text only (Default: "markdown") - `wait_for` (number, optional): Wait time in milliseconds for JavaScript-rendered content from 0 to 10000 (Default: 0) - `remove_scripts` (boolean, optional): Remove JavaScript and CSS from output (Default: true) - `follow_redirects` (boolean, optional): Follow HTTP redirects automatically (Default: true) **Example:** ```bash curl -X POST "https://connect.mcp360.ai/api/v1/web-scraping/scrape_page" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://techcrunch.com/2024/03/15/latest-ai-news", "format": "markdown", "remove_scripts": true }' ``` **Response Format:** ```json { "url": "https://techcrunch.com/2024/03/15/latest-ai-news", "title": "Latest AI News and Updates", "content": "# Main Article Title\n\nArticle content in markdown format...", "format": "markdown", "text_length": 5420, "images": ["https://example.com/image1.jpg", "https://example.com/image2.jpg"], "links": ["https://related-article.com", "https://source.com"], "meta": { "description": "Latest news about artificial intelligence...", "author": "John Doe", "published_date": "2024-03-15" } } ``` **Common Use Cases:** **Extract Blog Article:** ```bash curl -X POST "https://connect.mcp360.ai/api/v1/web-scraping/scrape_page" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://medium.com/@author/article-slug", "format": "markdown" }' ``` **Scrape Product Page:** ```bash curl -X POST "https://connect.mcp360.ai/api/v1/web-scraping/scrape_page" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://store.example.com/products/item-123", "format": "html" }' ``` **Extract News Article:** ```bash curl -X POST "https://connect.mcp360.ai/api/v1/web-scraping/scrape_page" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://news.example.com/technology/ai-breakthrough", "format": "text", "remove_scripts": true }' ``` **Scrape JavaScript-Heavy Site:** ```bash curl -X POST "https://connect.mcp360.ai/api/v1/web-scraping/scrape_page" \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://spa-website.com/content", "format": "markdown", "wait_for": 3000 }' ``` ## Authentication ```bash -H "Authorization: Bearer YOUR_API_KEY" ``` ## Rate Limiting - Page scraping (scrape_page): 1 credit per request - Rate limit: 500 requests per hour - Concurrent requests: Up to 5 parallel calls - Maximum page size: 10MB ## Support - **Blog & Tutorials**: https://mcp360.ai/blog - **Contact**: https://mcp360.ai/contact
These capabilities are available through the MCP protocol interface.

Available Tools

Technical specifications for the 1 available protocol tools.

1 tools

scrape_page tool

Input Specification
{
  "type": "object",
  "required": [
    "url"
  ],
  "properties": {
    "url": {
      "type": "string",
      "format": "uri",
      "description": "URL to scrape"
    },
    "render_js": {
      "type": "boolean",
      "default": false,
      "description": "Render JavaScript content (default: false)"
    },
    "country_code": {
      "type": "string",
      "default": "us",
      "description": "Country code for geo-located requests (default: 'us')"
    },
    "return_page_markdown": {
      "type": "boolean",
      "default": true,
      "description": "Return content as markdown (default: true)"
    }
  },
  "additionalProperties": false
}
Output Response
{
  "type": "object",
  "properties": {
    "url": {
      "type": "string"
    },
    "content": {
      "type": "string"
    },
    "success": {
      "type": "boolean"
    },
    "is_markdown": {
      "type": "boolean"
    },
    "status_code": {
      "type": "number"
    }
  }
}

Direct API Access

Production-ready REST endpoints for custom integrations.

POSTServer Metadata
/api/v1/web-scraping
curl "https://connect.mcp360.ai/api/v1/web-scraping" \
  -H "Authorization: Bearer YOUR_API_KEY"
POSTExecute Tool
/api/v1/web-scraping/{tool_name}
curl -X POST "https://connect.mcp360.ai/api/v1/web-scraping/scrape_page" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://example.com"
}'
Authentication

To authenticate, include your API key in the Authorization header using the Bearer scheme. Alternatively, you can use the X-API-KEY header.

Infrastructure

Unified MCP Gateway

One hub for 100+ production-ready tools with centralized management.

Unified API Key

Access 100+ MCP servers with a single authentication token.

Instant Testing

Test and debug any MCP server instantly in our integrated environment.

Centralized Billing

One monthly subscription for all your AI tool integrations and credits.

Scenarios

Standard Use Cases

Automated research and data gathering
Real-time monitoring and alerting systems
Content generation with current information
Competitive analysis and market intelligence
Decision support with live data feeds
API integration for AI applications

Frequently Asked Questions

What is Web Scraping Tools?

Web Scraping Tools is an MCP server that provides structured access to web scraping tools capabilities through a standardized protocol, enabling AI models to retrieve and process real-time data.

Which AI models are supported?

Any model that supports MCP protocol including Claude (via Claude Desktop), GPT-4, Gemini, and open-source LLMs through compatible frameworks.

How do I authenticate?

The server supports OAuth 2.0 authentication with API keys. You'll receive credentials upon registration which can be configured in your MCP client.

Is there rate limiting?

Yes, rate limits apply based on your subscription tier. Free tier includes generous limits for development, with higher limits available in paid plans.

Can I use this in production?

Absolutely. Web Scraping Tools is designed for production use with enterprise-grade reliability, security, and performance.

Deploy AI agent with Web Scraping Tools today.

Start building production-ready AI agent integrations in minutes with standardized protocol access.

Get Started FreeDocumentation
Enterprise Ready
Secure OAuth
24/7 Support