Iris — The Agent Eval Standard for MCP

Know whether your AI agents are actually good enough to ship. Iris is an open-source MCP server that scores output quality, catches safety failures, and enforces cost budgets across all your agents. Any MCP-compatible agent discovers and uses it automatically — no SDK, no code changes.

The Problem

Your agents are running in production. Infrastructure monitoring sees 200 OK and moves on. It has no idea the agent just:

• Leaked a social security number in its response
• Hallucinated an answer with zero factual grounding
• Burned $0.47 on a single query — 4.7x your budget threshold
• Made 6 tool calls when 2 would have sufficed

Iris evaluates all of it.

What You Get

Requires Node.js 20 or later. Check with node --version.

Quickstart

Add Iris to your MCP config. Works with Claude Desktop, Cursor, Windsurf, and any MCP-compatible agent.

{
  "mcpServers": {
    "iris-eval": {
      "command": "npx",
      "args": ["@iris-eval/mcp-server"]
    }
  }
}

That's it. Your agent discovers Iris and starts logging traces automatically.

Turn on the dashboard

Iris ships with a real-time web dashboard showing traces, eval results, cost breakdowns, and rule pass-rates. It's off by default so the MCP server stays lightweight — flip it on with a flag.

{
  "mcpServers": {
    "iris-eval": {
      "command": "npx",
      

... [View full README on GitHub](https://github.com/iris-eval/mcp-server#readme)