Web Content Extractor MCP Server (Agent-Optimized)

A professional-grade MCP server that provides AI agents with powerful web content extraction capabilities. Built specifically for the agent economy by Agenson Horrowitz.

🤖 Why This Exists

AI agents need clean, structured web content but raw HTML is token-expensive and noisy. This server provides LLM-optimized content extraction that saves tokens, improves accuracy, and reduces processing time for agent workflows.

⚡ Key Features

• Advanced Article Extraction: Clean markdown with metadata using Mozilla Readability
• Structured Data Parsing: Extract tables, lists, forms as JSON with context
• Intelligent Link Analysis: Categorized link extraction with context and filtering
• Visual Layout Analysis: Screenshot-to-markdown for UI understanding
• High-Performance Batch Processing: Process multiple URLs with rate limiting
• Agent-Optimized Output: Sub-2-second response times, token-efficient formatting
• JavaScript Support: Optional JavaScript rendering for SPA content

🚀 Installation

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "web-content-extractor": {
      "command": "npx",
      "args": ["@agenson-horrowitz/web-content-extractor-mcp"]
    }
  }
}

Cline Configuration

Add to your Cline MCP settings:

{
  "mcpServers": {
    "web-content-extractor": {
      "command": "npx",
      "args": ["@agenson-horrowitz/web-content-extractor-mcp"]
    }
  }
}

Via npm

npm install -g @agenson-horrowitz/web-content-extractor-mcp

Via MCPize (One-click deployment)

Deploy instantly on MCPize with built-in billing and authentication.

🛠️ Available Tools

1. extract_article

Extract clean article content as agent-optimized markdown.

Perfect for: News articles, blog posts, documentation, research papers

Features:

• Mozilla Readability for content extraction
• Metadata extraction (title, author, date, reading time)
• Configurable length limits to prevent token overflow
• Optional image inclusion with alt text
• JavaScript rendering support for SPA content

Example:

{
  "url": "https://example.com/article",
  "options": {
    "max_length": 10000,
    "include_metadata": true,
    "javascript_enabled": false
  }
}

2. extract_structured_data

Extract structured data (tables, lists, forms) as JSON.

Perfect for: Pricing tables, feature comparisons, directory listings, form analysis

Supported data types:

• Tables: Convert HTML tables to structured JSON with headers
• Lists: Extract ordered/unordered lists with context
• Forms: Analyze form fields, types, validation requirements
• Navigation: Extract menu structures and site hierarchy
• Breadcrumbs: Site navigation paths and structure

Example:

{
  "url": "https://example.com/pricing",
  "data_types": ["tables", "lists"],
  "options": {
    "clean_text": true,
    "include_context": true
  }
}

3. extract_links

Get all links with intelligent categorization and context.

Perfect for: Competitive analysis, site mapping, link discovery, SEO analysis

... View full README on GitHub