Krauncher

Run your training script on a remote GPU. Nothing more.

Krauncher is a minimal Python library for researchers who have a working local script and need a GPU — not a platform.

Website & API keys: krauncher.com

Quickstart

pip install krauncher
export CAS_API_KEY="cas_..."        # krauncher.com → Account → API Keys

Requires Python 3.11+.

import asyncio
from krauncher import KrauncherClient

client = KrauncherClient()           # reads CAS_API_KEY / CAS_BROKER_URL from env or .env

@client.task(vram_gb=1, timeout=120)
def multiply(size: int):
    import numpy as np               # imports go INSIDE the function
    a, b = np.random.rand(size, size), np.random.rand(size, size)
    return {"mean": float((a @ b).mean())}

async def main():
    handle = await multiply(size=1000)   # submit → TaskHandle
    print("task:", handle.task_id)
    result = await handle                # await the handle → TaskResult
    print("output:", result.output)
    print("gpu:", result.actual_gpu, "·", f"{result.execution_time_sec:.1f}s")

asyncio.run(main())

The decorated function becomes async: calling it submits the task and returns a TaskHandle; awaiting the handle (or await handle.wait(...)) returns a TaskResult.

Using an LLM / coding agent? Read AGENTS.md — a single accurate reference of the API, parameters, result fields, errors and constraints. Runnable examples live in tutorial/.

The problem with serverless ML platforms

Serverless orchestration platforms are genuinely impressive pieces of infrastructure. They handle container builds, secret management, artifact storage, scheduling, persistent volumes, and team dashboards.

They also charge you for all of it — whether you use it or not.

If you're fine-tuning a small model, running ablations, or iterating on a research experiment with a dataset under 2 GB, you're likely paying for an orchestration layer you don't need.

Krauncher does less, on purpose. It runs your existing Python function on a remote GPU, returns the result, and gets out of the way.

What Krauncher is (and isn't)

Good fit:

• Fine-tuning, LoRA, small-scale experiments with training datasets up to ~2 GB
• Researchers who already have a working local script
• Anyone tired of rewriting their code to fit a platform's abstractions
• Teams where "infrastructure" means one person and a credit card

Not the right tool if:

• You need managed versioned artifact storage
• Your team requires persistent shared volumes across runs
• Your dataset is hundreds of GBs with complex multi-node sharding
• You want a UI dashboard for experiment tracking

How it works

Add a decorator. Await your function. Get a result. Your existing code doesn't change — no base images, no volume mounts, no platform imports.

import asyncio
from krauncher import KrauncherClient

client = KrauncherClient()

@client.task(gpu_name="RTX4090", group_id="mistral-run", timeout=3600)
def finetune():
    from transformers import AutoModelForCausalLM, Trainer, TrainingArguments
    from datasets import load_dataset

    # Weights download to worker storage on first run (~15 GB for 7B);
    # later runs in the same group_id reuse the cached weights.
    model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")
    dataset = load_dataset("tatsu-lab/alpaca", split="train[:2000]")

    # ... your training logic, unchanged from local ...

    model.save_pretrained("/tmp/output")
    # Worker storage is ephemeral — sync checkpoints out before returning.
    upload_to_s3("/tmp/output", "my-checkpoints/run-1")
    return {"status": "done", "checkpoint": "s3://my-checkpoints/run-1"}

async def main():
    result = await finetune()        # submit and wait
    print(result.output)

asyncio.run(main())

The decorated function is async — always call it from an async context

... View full README on GitHub