How do I install LLM Eval Harness?

Copy the stdio config for Cursor, Claude Desktop, Claude Code, or VS Code on this page. It uses the publisher's install, not a PlugMCP proxy.

Does LLM Eval Harness need an API key?

The publisher lists these environment variables: ANTHROPIC_API_KEY, GEMINI_API_KEY.

What transport does LLM Eval Harness use?

LLM Eval Harness runs locally over stdio.

When was LLM Eval Harness last checked?

Last checked 2026-11-10. Run a golden set and gate LLM model swaps on accuracy, with latency and cost per request.

LLM Eval Harness MCP server

Power your agent

Add this server

The publisher's own command. Not a PlugMCP endpoint.

Add to Cursor Add to VS Code Run on PlugMCP

Host a copy with our keys and logs.

Local command

docker run -i --rm ghcr.io/razilzul/llm-eval-harness:1.0.0

Credentials from the publisher

ANTHROPIC_API_KEY — Only needed to evaluate anthropic:* models. Not needed for mock models.
GEMINI_API_KEY — Only needed to evaluate gemini:* models. Not needed for mock models.

Claude Code has no install link for a local command. Copy the command below and run it in a terminal.

shClaude Code

1ANTHROPIC_API_KEY=<secret> GEMINI_API_KEY=<secret> claude mcp add io.github.razilzul-llm-eval-harness -- docker run -i --rm ghcr.io/razilzul/llm-eval-harness:1.0.0

Run in a terminal.

jsonClaude Desktop

1{2  "mcpServers": {3    "io.github.razilzul-llm-eval-harness": {4      "command": "docker",5      "args": [6        "run",7        "-i",8        "--rm",9        "ghcr.io/razilzul/llm-eval-harness:1.0.0"10      ],11      "env": {12        "ANTHROPIC_API_KEY": "<secret>",13        "GEMINI_API_KEY": "<secret>"14      }15    }16  }17}

claude_desktop_config.json

Local · ociPublisher installStep-by-step: add this in Claude

Remote URL or a command on your machine? How the two installs differ

Run on PlugMCP

Host a copy with our keys and logs.

A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.

Launch

Server facts

Local

Transport: stdio
Version: 1.0.0

Tools

No tool list yet. Remote servers get one after the first health check.

FAQ

LLM Eval Harness

Add this server

Host a copy with our keys and logs.

Server facts

Tools

Questions

How do I install LLM Eval Harness?

Does LLM Eval Harness need an API key?

What transport does LLM Eval Harness use?

When was LLM Eval Harness last checked?

Documentation