Observability
Usage, cost, savings, and optimizer decisions in the console.
How it works
The gateway writes usage to your Postgres, in-process: no extra service, no project keys.
The console (:3000) is a read-only view behind the gateway's admin gate
(ANYRAY_ADMIN_TOKEN, or an optional
role-scoped SSO session). It never touches
Postgres directly, only the gateway's /admin/* endpoints, so the browser never sees the
database, and stored rows carry metadata, not content.
Console pages
With SSO on, any role can read the dashboards and traces. Changing settings needs a higher role,
and request/response bodies in a trace open only for security_admin/owner
(observability:read-content); every other
role sees the same trace with the bodies hidden, metadata and optimizer decisions intact.
| Page | Shows |
|---|---|
| Home | Spend, savings, and traffic across the org: the headline KPIs and token charts. |
| Usage | Per-user token use over time, with each user's monthly cap and a split of their traffic by upstream provider. |
| Users | Enroll users, set USD budgets, manage SSO/MDM, and diagnose auth failures. |
| Traces | Request list (including the provider and reasoning effort of each request) + per-request drill-down (optimizer decisions, provider attempts). |
Spend and savings
The Home dashboard reads GET /admin/spend/dashboard. Its headline numbers:
| KPI | What it means |
|---|---|
| Tokens saved | Input tokens the optimizer trimmed. |
| Total saving | Trimmed tokens as a share of the cost-weighted total (trimmed plus kept), with tokens served from the provider's cache counted at roughly a tenth of a fresh token, the share of a fresh token they cost. |
| Value created | The list-price value of what the optimizer saved (grossSavingsUsd, before over-allowance scaling), split into what a subscription covers vs. what's billable. Each trim is priced on its cache evidence: proven already-cached is worth a tenth of fresh input; no cache evidence counts at the full list input rate. On a warm session the token count can climb while the dollar value moves little. |
| Extra capacity | How much further the same subscription usage limits stretch: trimmed tokens against what the seat actually consumes, with provider-cached tokens weighted at roughly a tenth (shown only when you have seats). |
The same response splits cost and tokens by model, team, user, client tool, and reasoning effort. Every row also carries a billing lane, read from quota response metadata, which sets how its cost and savings count:
| Lane | Seat? | Cost & savings |
|---|---|---|
api-key | No | Full cost, full savings. |
subscription (included) | Yes | Configured over-allowance fraction. |
subscription-extra (provider-confirmed) | Same seat | Full real cost and savings. |
To check reported savings against real measurements, sample a shadow baseline or run an audited holdout.
Savings accuracy
Saved-token figures come from a characters-to-tokens estimate, fast enough to run inline but not how a provider bills. With shadow baseline sampling on, the gateway re-counts each sampled trim with a real tokenizer and records the gap on the spend row.
These are Anyray's own diagnostic about optimizer accuracy, so your console does not show them. They are aggregate-only, carry no per-request or per-user shape and no content, and are never priced. The audit runs in-process after the response reaches the client, so it costs no request latency and no extra upstream call.
Usage and caps
The Usage page (GET /admin/spend/usage) shows a live cap gauge (used / cap / percent) for
capped users. Set USD budgets on the Users page; token caps are set through
PUT /admin/user-caps. When a capped user crosses a share of their cap (default 80% and
100%), the gateway posts one alert to a Slack-compatible webhook and logs a security event,
once per user, threshold, and month across the whole fleet.
Traces
A trace records one request end-to-end; bodies follow the content mode:
| Row | What it captures |
|---|---|
| Trace | The request: session, user, status, cost, token totals, the provider that served it, and the reasoning effort it was sent with. |
| Span | Each optimizer decision: kind, summary, tokens/cost saved. |
| Generation | Each provider attempt: model, usage, timing (one per retry). |
The provider is the resolved slug (anthropic, bedrock, vertex-ai), which the model id cannot
tell you: the same claude-sonnet-4-5 runs on Anthropic direct, Bedrock, or Vertex. Effort is a
column and a filter, showing a dash when the request named none. Full field list:
Gateway → Spend and metering.
Traces are the only store that can hold content, and only when the content mode keeps it. Retention is set on the console Privacy page, and defaults to 90 days. A GDPR delete erases a user's traces.
Where it's stored
Everything derives from Postgres; the gateway is the only writer.
| Store | Table(s) | Read by |
|---|---|---|
| Spend | anyray_spend | /admin/spend/dashboard · /admin/spend/usage |
| Trace | anyray_traces + anyray_observations | trace drill-down (/admin/observability/*) |
| Auth events | anyray_auth_events | /admin/auth-events |