web-to-markdown-mcp

An MCP server that fetches a URL and returns the main content as clean Markdown. Uses plain HTTP when possible and real Chromium when needed.

Why

Most MCP web-fetch tools either:

• use plain HTTP, which fails on JS-required pages and gets blocked by Cloudflare bot detection on many sites; or
• use a real browser but return raw HTML or accessibility-tree snapshots, which are noisy and token-heavy when you just want to read the article.

This server uses a three-tier strategy, using the fastest approach that works for each URL:

1. Native markdown fast-path. Every request first tries a plain HTTP GET with an Accept: text/markdown header. Servers that support content negotiation — such as Cloudflare-hosted sites with Markdown for Agents enabled — respond with Content-Type: text/markdown, and the body is returned immediately with no browser overhead.

2. Plain HTTP + extraction. If the server returns HTML, trafilatura extracts the article body as clean Markdown directly from the static response. Heuristics detect JS shells — pages that return only a "JavaScript required" stub in the static HTML — and fall through to tier 3 for those.

3. Headless browser. When tiers 1 and 2 yield nothing usable, patchright (a Playwright fork with anti-detection patches) drives real Chromium and trafilatura extracts the rendered content. A single headless browser instance is kept alive across calls — it launches lazily on the first browser-tier fetch and stays running for the session, so subsequent fetches pay only navigation time rather than Chromium startup cost (~2–5 s).

For sites that block even headless patchright (aggressive bot detection), pass headless=False. This uses a visible Chromium window — slower and visually intrusive, but clears most remaining challenges. A headed browser is launched and closed for each fetch that needs it; it is not kept persistent since headed fetches are an infrequent escape hatch rather than the common case.

After browser navigation, the server polls the DOM and runs trafilatura, returning as soon as two consecutive polls produce the same extraction. This means it returns within a few hundred milliseconds for typical pages — rather than waiting for analytics, ads, and other late-loading resources to finish — and gives slow SPAs and bot-challenge clearance time to settle without timing out prematurely.

For a typical article, expect roughly 80% fewer tokens than the raw HTML and roughly 90% fewer than a full accessibility-tree snapshot.

Installation

Requires Python 3.10+ and a one-time Chromium download (~300 MB).

# Run directly with uv (no install step)
uvx web-to-markdown-mcp

# Or install with pip
pip install web-to-markdown-mcp

# One-time browser download
patchright install chromium

MCP client configuration

Claude Desktop

Edit your config file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "web-to-markdown": {
      "command": "uvx",
      "args": ["web-to-markdown-mcp"]
    }
  }
}

Restart Claude Desktop.

LM Studio

Edit ~/.lmstudio/mcp.json (Developer tab → Edit mcp.json) — same JSON block as above. Then enable Allow calling servers from mcp.json in the Developer tab's Server Settings. The server appears in the Integrations tab of any new chat.

Cursor / Windsurf / VS Code

Same JSON block, in each client's MCP config location.

Claude Code

claude mcp add web-to-markdown -- uvx web-to-markdown-mcp

Usage

The server exposes a single tool:

fetch_url_as_markdown

| Parameter | Type | Default

... View full README on GitHub