harvest-mcp

Plain-English topic in → 12 experts fan out over Hugging Face + GitHub in parallel → coverage-scored training pack JSON out in ~30 seconds.

No API keys. No Bridge. No account. Works out of the box once connected to any MCP host — no LLM inside; your agent brings the thinking.

Harvest is a training-data scout: you describe what you want to train on in plain English ("image classification", "medical question answering"), and 12 domain experts — code, math, science, language, vision, audio, medical, law, knowledge, safety, affect, systems — each sweep Hugging Face datasets and GitHub repos with their own curated queries. The local mixer then composes the hits into a balanced, coverage-scored training library pack (flare-harvest-library v1 JSON) you can hand to any training pipeline.

Install

npx github:The-Warl0ck/harvest-mcp

Or add it to Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "harvest": {
      "command": "npx",
      "args": ["github:The-Warl0ck/harvest-mcp"]
    }
  }
}

With optional tokens for higher rate limits:

{
  "mcpServers": {
    "harvest": {
      "command": "npx",
      "args": ["github:The-Warl0ck/harvest-mcp"],
      "env": {
        "HF_TOKEN": "hf_...",
        "GITHUB_TOKEN": "ghp_..."
      }
    }
  }
}

Requirements: Node.js ≥ 20.

Example session

You: find me training data for a small vision-language model

harvest.search { "topic": "vision language instruction tuning" } → ~40 hits: LLaVA-NeXT data, the Cauldron, DOCCI, LLaVA repo… plus the 12-expert atlas sweep for breadth (code, math, safety…)

harvest.compose { "goal": "small vision-language model", "catalog": […] } → { "title": "small vision-language model", "picks": [ { "id": "lmms-lab/LLaVA-NeXT-Data", "expert": "vision", "why": "Anchor Vision on Hugging Face" }, … ], "rationale": "Local mixer staged 22 items across 12/12 experts. No mouth required.", "engine": "local" }

harvest.pack { "goal": "small vision-language model", "name": "VLM Mix v1", "catalog": […] } → flare-harvest-library v1 JSON with coverage: { filled: 12, totalExperts: 12, balance: 0.94 } and per-item ingest: { type: "hf-dataset" | "github-repo", ref } pointers.

The second half — the hands. Packs start as pointers; pull turns them into files:

harvest.resolve { "item": { "id": "openai/gsm8k", "source": "huggingface" } } → the loot preview: 4 parquet shards + README, each with a one-line reason, nothing stupid

harvest.pull { "pack": {…}, "destination": "download" } → resolves + fetches every item, updates the pack's index stanza, and builds VLM Mix v1.harvest.zip — one file with everything plus MANIFEST.json provenance

harvest.push { "bundle": "/path/to/VLM Mix v1.harvest.zip", "owner": "you", "repo": "my-data", "token": "ghp_…", "create_if_missing": true } → one commit on a new private repo: files + manifest, message says what it is and where it came from

The same loop works for code. Second session — the build pack:

You: I want the rate-limiting code from a few good repos, packaged so my agent can build with it

harvest.search { "topic": "express rate limiting middleware" } → the code expert surfaces the repos with the parts you want

harvest.pack { "goal": "add rate-limiting to my API", "name": "Rate Limit Parts v1", "catalog": […], "purpose": "code" } → flare-harvest-codepack v1 JSON — same item envelope, no training-coverage scoring

harvest.pull { "pack": {…}, "destination": "download", "kinds": ["code", "docs"] } → pulls just the source files (skips weights/datasets), builds the .harvest.zip

harvest.push { "bundle": "…", "owner": "you", "repo": "rate-limit-parts", "create_if_missing": true } → your agent pulls the repo and builds from the parts

Tools

... View full README on GitHub