Observability

Usage, cost, savings, and optimizer decisions in the console.

How it works

The gateway writes usage to your Postgres in-process. The console (:3000) is read-only and calls only the gateway's /admin/* endpoints, behind ANYRAY_ADMIN_TOKEN or an optional role-scoped SSO session.

Console pages

Page	Shows
Home	Org-wide spend, savings, traffic: headline KPIs and token charts.
Cockpit	Per-user token use over time, by model and upstream provider. Open a person for their Activity: the days and hours they used AI tools, and their usual hours in your time zone. Beside it, a gauge shows the token value they got per $1 of their subscription seat, against a monthly plan price you enter. The old /usage route redirects here.
Users	Enroll users, set USD budgets, manage SSO/MDM, diagnose auth failures.
Sessions	One row per conversation; open one to search and filter its requests, then a request's optimizer decisions and provider attempts. An open Claude Code session shows its subagents' request count and share of list cost.

Role (table)	Sees
viewer	Own traces and those of agents assigned to them
auditor and up	Dashboards and everyone's traces
security_admin, owner	Also request/response bodies (observability:read-content)

Spend and savings

Home reads GET /admin/spend/dashboard:

KPI	Meaning
Tokens saved	Input tokens saved, the days of the Savings chart added up. Each day is that day's Total saving rate applied to the tokens it billed. The line under it says how many of them were already in the provider cache (worth about a tenth of a fresh token); it stays blank until the gateway measures it.
Total saving	(untreated cost − actual cost) / untreated cost, summed in dollars over the window and floored at zero. The untreated cost re-prices each session as the provider would have cached your tools' own requests, on both automatic (OpenAI) and breakpoint (Anthropic, Bedrock) caches. Every turn an optimization ran on is then settled the same way: it gains or loses the gap between its cache-read share and that of the same model and tool's turns without one, priced at that model's cache-read discount. Today's turns use the last 24 hours as the comparison and are updated every 15 minutes; when the day ends they are re-priced against that whole day, or keep their value if the day had too few turns to compare. A connect hook trim counts the cache write on the next request plus a cache read on each later request so far, and keeps counting until a gap of more than an hour or a compaction. The sparkline plots the same ratio per day.
Value created	Untreated cost − actual cost over the window, floored at zero: the dollars behind Total saving, before over-allowance scaling.
Extra capacity	Subscription-seat traffic's avoided cost over its actual cost: how much more work each seat now does. Shown when seats saved. A gateway that predates session savings shows trimmed tokens vs what seats consume (cached at about a tenth).

It also splits by model, team, user, client tool, and reasoning effort. cache.byStrategyClass groups warm same-session turns by the class of strategy that fired and counts, per class, the turns whose cached prefix was re-written and how many of those a mint had priced (rewriteTurns, rewriteTokens, pricedTurns, pricedRewriteTurns), read against the none baseline. Billing lane (from quota response metadata):

Lane	Seat?	Cost & savings
api-key	No	Full cost, full savings.
subscription (included)	Yes	Configured over-allowance fraction.
subscription-extra (provider-confirmed)	Same seat	Full real cost and savings.

Verify with a shadow baseline or an audited holdout.

Usage and caps

GET /admin/spend/usage returns per-user usage and a monthly cap gauge (used / cap / percent). USD budgets: Users page. Token caps: PUT /admin/user-caps (usage governance). At 80% and 100% of a cap (defaults) the gateway posts one Slack-compatible webhook alert and logs a security event, once per user, threshold, and month fleet-wide.

Traces

Filter by Batch read under Strategies on Sessions or an open session, including requests where it was the only strategy credited. The request's transformation breakdown shows its saved tokens and avoided round trips.

Row	Captures
Trace	Session, user, status, cost, token totals, serving provider, reasoning effort sent, thinking-binding outcome.
Span	Each optimizer decision: kind, summary, tokens/cost saved.
Generation	Each provider attempt: model, usage, timing (one per retry).
Input breakdown	What the sent prompt was made of: tools, system prompt, <system-reminder> blocks, history and the current turn, plus cache markers and what Anyray removed, deferred or added.

The input breakdown shows in every content mode, because it holds sizes and never text. On a trace it is the Request breakdown treemap under the Content card: one tile per kind, sized by its estimated share of the input the request carried. Hover a tile for its tokens and share. Codex traces get it too; a Codex turn that continues an earlier response sends only its new items, so its breakdown covers that turn's new input, not the whole conversation.

In plaintext mode the Content card opens in Raw, the stored text unchanged. Structured reads the request as a conversation, one folded entry per message; click an entry to open it. The trace store keeps the start and end of a long request, so the first entry also holds the part it dropped and the end of the message it cut. The console remembers your last pick in this browser.

security_admin and owner also get Export content on Sessions: one CSV file of the bodies of every request in the current range and filters, up to 10,000. Excel holds at most 32,767 characters in a cell, so read long bodies from the API.

Where it's stored

The gateway is the only writer.

Store	Table(s)	Read by
Spend	anyray_spend	/admin/spend/dashboard · /admin/spend/usage
Trace	anyray_traces + anyray_observations	trace drill-down (/admin/observability/*)
Auth events	anyray_auth_events	per-user Diagnose section (/admin/auth-events)
Replica health	anyray_replica_health	Health page (/admin/health → replicas[])