Install

Three ways in. They share the same rules and the same report, so you can start in the browser and move up later.

Step 1

Browser

Nothing installed, nothing uploaded. Reads the session logs already on your machine.

Open the evaluator and drop your ~/.claude/projects/ folder, or a few session files. You get tasks, time, cost and friction per session. The numbers are read from past logs, so they are labelled as such.

Open the evaluator

Step 2

Local

Live telemetry from Claude Code into a collector on your machine. Needs Docker.

A pip install package is not published yet. For now, run it from the repository.

Start the stack

git clone https://github.com/KrishantM/project-eval.git
cd project-eval
docker compose up -d

This starts Postgres, Redis, the api and the grading worker. The api and this dashboard are on http://localhost:8000. The judge uses Ollama on your machine by default (ollama pull qwen3:8b).

Point Claude Code at it

Add this to ~/.claude/settings.json (merge it with any env block you already have), then start a new Claude Code session.

~/.claude/settings.json

{
  "env": {
    "CLAUDE_CODE_ENABLE_TELEMETRY": "1",
    "OTEL_LOGS_EXPORTER": "otlp",
    "OTEL_METRICS_EXPORTER": "otlp",
    "OTEL_EXPORTER_OTLP_PROTOCOL": "http/json",
    "OTEL_EXPORTER_OTLP_ENDPOINT": "http://localhost:8000/otlp"
  }
}

Prompt text is not sent unless you also set OTEL_LOG_USER_PROMPTS=1, and the server drops your email and account ids before storing anything. The setup page shows when the first task arrives.

Optional: import past sessions

Reads the logs the browser evaluator reads, into the database, so the history is in one place.

docker compose run --rm -v ~/.claude/projects:/import:ro api \
  python -m eval_server.cli import-claude-code --root /import

Step 3

Team

Your own cluster. A default-deny egress policy keeps task data inside it.

Install the chart

helm install project-eval ./deploy/helm/project-eval -n project-eval --create-namespace

Run it from a clone of the repository, after pushing the image to a registry your cluster can pull from. It creates the api and worker Deployments, Postgres and Redis StatefulSets, a migration Job, and the egress NetworkPolicy.

Set your values

my-values.yaml

# my-values.yaml
image:
  repository: registry.example.internal/project-eval
  tag: "1.0.0"
api:
  publicUrl: "https://eval.example.internal"   # how agents reach the api
judge:
  model: "ollama:qwen3:8b"                     # any LangChain init_chat_model id
ingestToken: "change-me"                       # every /v1 and /otlp call then needs a bearer token
postgres:
  password: "change-me"
keda:
  enabled: true                                # scale workers on the job stream (needs KEDA)
networkPolicy:
  enabled: true
  judgeEgress: [{cidr: 10.0.0.12/32, port: 11434}]   # the only egress besides DNS

helm upgrade --install project-eval ./deploy/helm/project-eval -n project-eval -f my-values.yaml

Connect agents

Use the settings block above with OTEL_EXPORTER_OTLP_ENDPOINT set to your api's address plus /otlp, and OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <token> when you set an ingest token. The dashboard's setup page fills in the address for you.