Measure on your traffic
Have Claude Code signed in and Anyray Connect installed. Claude checks the kit and gateway credentials before planning. Keep credentials in the environment, never in chat or scenario files.
The console's Measure on your traffic page shows published results for any reported client.
Everyone sees the results they published; auditor and up see the whole workspace. Its bundled
test workflow executes Claude Code only; open Run the Claude Code test kit. Viewing results
needs your console sign-in. Billing workspace membership does not grant access.
Scenario files, patches, the plan lock and run records stay on your machine. Paid runs send your task to the model provider, and the Anyray arm goes through your gateway, which records it like any other traffic under your content mode (encrypted by default). Savings on traffic your client-test key signs count as value created and are never invoiced.
Publishing sends only a redacted result to your frozen gateway, and only when you ask. It carries no task content, repository URLs, transcripts, free text or the plan lock's nonce.
In Claude Code, run:
/plugin marketplace add anyrayHQ/benchmarks
/plugin install anyray-test@anyray-benchmarks
From your repository, ask Claude to measure Anyray on the task, script or test description you already planned to run. Include the test itself in your message.
Claude records your words unchanged as source: client under anyray-test/client-1/.
It proposes extra scenarios separately. Choose which proposals to include; your test stays.
Supply your budget when asked. The noise rounds measure what one session costs.
Before any spend, the plan shows each scenario's source and prediction, and the spend ceiling
(spendCeilingUsd), which is your budget. The pair count and the saving it can detect come
after the noise rounds measure your session cost.
Short tasks are predicted at parity. A plan can also show prediction: none with a reason.
A prediction is not a measured saving.
Review the scenarios, pinned versions and ceiling, then approve freezing the plan. Claude freezes and verifies it and shows its hash. Any later change needs a new plan.
Claude shows the spend ceiling again. Choose Start explicitly for that frozen plan. Approving the plan does not start paid work.
Claude measures direct-versus-direct noise first, then runs the paired tests. It reports progress, exclusions and costs when available. A larger scope or ceiling needs a new plan and another Start.
Read the kit's exact verdict and pooled Rule 0 result, including exclusions and predicted
versus actual results. INCONCLUSIVE stays INCONCLUSIVE; a median ratio alone is not a
savings claim.
Claude creates a local HTML report from the validated, redacted result. Keep raw transcripts local.
Ask Claude to publish this result to your console. It sends the validated, redacted result only to the frozen gateway and returns a result id. If the console is unavailable, Claude reports the blocker.
Open /console/measure to see published workspace receipts. Each frozen plan keeps its own
paired session costs and cost-check verdict. Test details holds the original verdict,
release and plan hash. The page refreshes every minute.
Reported inference costs use list prices. Provider invoices and subscription cash savings are not measured here.
Your check shows how many pairs passed your own check on each arm, and a warning names any pair that passed only direct. Results from kits before 0.3.0 carry no check counts.
If your test ran kinds still in shadow on your tenant, the receipt says how many. Your live traffic gets them only after they are promoted. Retries and separate overhead are not reported; check your local run records for those. A published result does not prove a new test has completed.
Optimizer → Measurement also shows the result beside the holdout reading and links to this flow.
See the command reference for manual commands, credentials and the optional per-release workflow.