Scrape, map, crawl, search and monitor the web. Runs locally with no API key, cites every source.
MCPpedia last refreshed this data
io.github.brainfish-ai/webcrawl is an MCP server that scrape, map, crawl, search and monitor the web. Runs locally with no API key, cites every source. Its tool list has not been published yet over http and stdio, requires no API key, and scores 64/100 on MCPpedia's security, maintenance and efficiency rubric.
Config is the same across clients - only the file and path differ.
{
"mcpServers": {
"io-github-brainfish-ai-webcrawl": {
"args": [
"-y",
"@webcrawl/mcp"
],
"command": "npx"
}
}
}Are you the author?
Add this badge to your README to show your security score and help users find safe servers.
Scrape, map, crawl, search and monitor the web. Runs locally with no API key, cites every source.
This server supports HTTP transport. Be the first to test it - help the community know if it works.
Five weighted categories - click any category to see the underlying evidence.
No known CVEs.
Checked @webcrawl/mcp against OSV.dev.
Be the first to review
Have you used this server?
Share your experience - it helps other developers decide.
Sign in to write a review.
Others in browser
Open-source web scraper for AI agents with scrape, crawl, and map tools
Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡
Search-first MCP server for JavaScript reverse engineering: 735 tools across 36 domains.
Firecrawl, the context API for AI agents: search, scrape, interact with the web, plus paper search.
MCP Security Weekly
Get CVE alerts and security updates for io.github.brainfish-ai/webcrawl and similar servers.
Start a conversation
Ask a question, share a tip, or report an issue.
Sign in to join the discussion.
Give your AI agent the whole web. Free, local, no API key.
Search, scrape, map, crawl and monitor any website, SPA, PDF, YouTube video, GitHub repo, ArXiv paper, product page or CVE, from Claude Code, Cursor, Codex or your terminal.
Every frame above is real output, recorded with node scripts/record-demo.mjs. No API key, no account.
npx skills add brainfish-ai/webcrawl-anything
That installs all eight Webcrawl Agent Skills into Claude Code, Cursor, Codex, Gemini, Windsurf or any agent that reads SKILL.md. Only want one? Pick it by name:
npx skills add brainfish-ai/webcrawl-anything --skill webcrawl-deal-watch
To also register the @webcrawl/mcp server, so your agent can call the crawl, search and monitor tools directly:
npx webcrawl setup
Restart your agent and paste one of these:
Is CVE-2024-3094 in anything I depend on? Look it up, give me the CVSS score, CISA KEV
status and fixed versions, then check my package-lock.json and link the evidence.
Map https://docs.stripe.com/webhooks, crawl every page under it into Markdown, and
write me a one-page cheat sheet with the source URL under each section.
Watch https://news.ycombinator.com every hour and alert me when a story about MCP
reaches the front page. Show me the monitor you created.
More reproducible prompts, each with the command it runs: docs/demos.md.
No agent? Use the CLI directly: npx webcrawl search "react 19 compiler".
| Target | Returns | Requirements |
|---|---|---|
Websites & JS SPAs (web) | Clean Markdown, semantic headings, tables, code blocks, links, OpenGraph/JSON-LD, and SHA-256 content hashes | Zero-key local (impit TLS impersonation + local Chrome) or webcrawl.app cloud stealth browsers |
PDF & Office Docs (pdf-office) | Page-numbered Markdown, tables, slide notes, and automatic OCR on scanned PDF pages (pagesOcr + honest pagesSkipped) | Built-in PDF inspector plus pdf-parse, mammoth, officeparser, and tesseract.js |
YouTube & Media (youtube-media) | Timestamped transcript Markdown, video title/channel/duration metadata, and spoken audio transcription | Built-in zero-key YouTube multi-client caption extractor; optional audio transcription provider for raw MP3/MP4 |
Shopping & Products (shopping) | Multi-retailer price comparison, currency-normalized offers, stock status, ratings, review counts, and s |