SnoopScan

Turn any public page into clean markdown, structured JSON, or a schema you define — including the pages that block everything else. An MCP server for AI agents, a REST API, and SDKs.

This repository is SnoopScan's open core. It runs on its own, and the hosted API at snoopscan.com adds the parts that are not here. See Open source vs hosted API.

Set up in your AI chat

Paste this into Claude Code, Codex, Cursor, VS Code or any AI agent:

Read and follow https://snoopscan.com/agent-onboarding/SKILL.md

Your agent reads the guide and does every step. When your browser opens, sign in or create a free account (1,500 credits a month, no card) and press Approve. That's it: ask it to read a page.

In Claude Code you can install the SnoopScan plugin instead, which brings SnoopScan's skills and its MCP server. Type these in Claude Code's chat:

/plugin marketplace add SnoopScan/snoopscan-core
/plugin install snoopscan@snoopscan

The Claude app and ChatGPT connect by signing in, with no key to copy: add a custom connector (Claude) or a developer-mode app (ChatGPT) with the address https://api.snoopscan.com/mcp-oauth. Steps for every client are at snoopscan.com/integrations.

Use it from your code

The examples below call the hosted API and read your key from SNOOPSCAN_API_KEY. Create one at snoopscan.com/app/keys, or let your agent fetch it for you with the setup above.

No key yet? /v1/scrape, /v1/search and /v1/parse also work with no Authorization header: 20 free calls a day per IP address, for pages that load without a browser. Every answer says how many free calls are left.

curl

curl -X POST https://api.snoopscan.com/v1/scrape \
  -H "Authorization: Bearer $SNOOPSCAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/", "formats": ["markdown", "links"]}'

Python

pip install snoopscan

import os
from snoopscan import SnoopScan

snoop = SnoopScan(api_key=os.environ["SNOOPSCAN_API_KEY"])
print(snoop.scrape("https://example.com/").markdown)

JavaScript / TypeScript

npm install snoopscan

import { SnoopScan } from 'snoopscan';

const snoop = new SnoopScan({ apiKey: process.env.SNOOPSCAN_API_KEY! });
console.log((await snoop.scrape('https://example.com/')).markdown);

Other languages

Every SDK has the same methods, the same error codes and the same retry rules, and reads SNOOPSCAN_API_KEY from the environment when no key is passed. Each directory's README covers every method.

Features

• Clean output: markdown, HTML, links, screenshots, or JSON in your own schema.
• Handles the hard pages: JavaScript rendering, blocked pages and proxies, with no setup (hosted API).
• Whole sites: crawl a site, map its URLs, or batch thousands of URLs as one job.
• Structured data: fill a JSON schema or a named template, with a confidence score per page.
• Stores and blogs in one call: a store's whole product catalog or a blog's posts, without a crawl.
• Pay for results: failed requests cost nothing, and every response carries a cost object.
• Agent ready: every endpoint is also an MCP tool.

Open source vs hosted API

This repository contains:

• the REST API
• ext

... View full README on GitHub