crawlport

crawlport is a doorway between an agent and the readable web, backed by Firecrawl. Give it one page and it hands back clean markdown; give it a whole site and it will map the pages, crawl a section, or scrape a list in batch; give it a question and it can search, pull structured fields out of pages to a schema you describe, research across many sources, or draft the llms.txt a site would publish about itself. It runs as a CLI or as an MCP server, and it needs a Firecrawl API key.

Setup

Get a key from Firecrawl and export it:

export FIRECRAWL_API_KEY=<redacted>

Without the variable set, every command and tool call fails with an error that names FIRECRAWL_API_KEY — nothing is sent anywhere.

Install

npx @reneturcios/crawlport scrape https://example.com/some/page

Or run the container image:

docker run -i -e FIRECRAWL_API_KEY ghcr.io/cloudcorprecords/crawlport

The container defaults to the MCP server (see below). Pass CLI arguments to use it as a CLI instead.

CLI

# One page, as markdown (add --formats html,links etc. for more)
crawlport scrape https://example.com/article --only-main-content

# Discover a site's URLs, optionally filtered
crawlport map https://example.com --search pricing --limit 100

# Web search, with locale controls
crawlport search "standing desk reviews" --limit 5 --lang en --country US

# Start a crawl, then poll it with the returned job id
crawlport crawl https://example.com/docs --limit 50 --max-depth 2
crawlport crawl-status <job-id>

# Structured extraction and batch scraping
crawlport extract https://example.com/a https://example.com/b --prompt "pull the price"
crawlport batch-scrape https://example.com/a https://example.com/b
crawlport batch-status <job-id>

# Research and llms.txt
crawlport deep-research "home battery storage costs in 2026" --max-urls 40
crawlport llmstxt https://example.com --max-urls 25

crawlport --version

Successful calls print the API's JSON response. Crawl and batch jobs are asynchronous upstream: the start call returns a job id, and the matching status command reads its progress and results. Exit codes: 0 on success, 1 when the API call fails (including a missing key), 2 for usage errors.

MCP server

One bin serves both faces. The mcp subcommand starts a stdio MCP server:

npx -y @reneturcios/crawlport mcp

MCP client configuration:

{
  "mcpServers": {
    "crawlport": {
      "command": "npx",
      "args": ["-y", "@reneturcios/crawlport", "mcp"],
      "env": { "FIRECRAWL_API_KEY": "your-key-here" }
    }
  }
}

Tools

... View full README on GitHub