Skip to content
Project Eval

How it works

From an agent's task to a number you can trust

Postgres holds the truth. Redis only carries jobs, and a job that cannot reach Redis waits in Postgres until it can. Every part below links to the file that implements it.

Hover or click any part or connection. Dashed means planned.

OTLP JSONOTLPPOST /v1/tasksone transactionSKIP LOCKEDXADDXREADGROUPgradesampledcomputed liveClaude CodeOpenTelemetryLiteLLM proxyplannedPython SDKexact boundariesapiFastAPIPostgressource of truth + outboxReportbasis + rangeOutbox relaypublish_outbox()Redis Streamseval:jobsGrading workerLangGraphHuman reviewinterrupt()

Send a task through

Follow one task from the agent to the report, step by step. Switch on Break Redis first to see the outbox hold the job until Redis returns.

api (FastAPI)

What it does

Validates every request against the schema, drops identity attributes (email, account and organisation ids), and drops prompt and output text unless the agent has content capture switched on. That rule is enforced here, not trusted to the sender.

POST /v1/tasks response (201 created, 200 updated)

{"task": {"id": "0b9e…", "external_id": "ticket-4812"},
 "created": true,
 "grade_job_id": "5d1a…",
 "grade_job_published": true}

Failure it handles

Bearer token compared with secrets.compare_digest, so timing reveals nothing. /readyz returns 503 when Postgres or Redis is unreachable, so Kubernetes stops sending traffic without restarting the pod.

On Kubernetes

The Helm chart runs the same pieces in one namespace. Data stays in the cluster: the only way out is the judge endpoint you allow.

namespace: project-eval