Observability

Usage, cost, savings, and optimizer decisions in the console — no content.

How it works

The gateway writes usage to your Postgres, in-process — no extra service, no project keys. The console (:3000) is a read-only view behind the gateway's admin gate (ANYRAY_ADMIN_TOKEN, or an optional role-scoped SSO session): it never touches Postgres directly, only the gateway's /admin/* endpoints — so the browser never sees the database, and stored rows carry metadata, not content.

Console pages

Every page sits behind the admin gate. With SSO on, any role can read the dashboards and traces; changing settings or viewing trace content needs a higher role.

Page	Shows
Home	Spend, savings, and traffic across the org — the headline KPIs and token charts.
Usage	Per-user token use over time, with each user's monthly cap.
Users	Enroll users, set token caps, manage SSO/MDM, and diagnose auth failures.
Traces	Metadata-only request list + per-request drill-down (optimizer decisions, provider attempts).

Request and response bodies in a trace open only for security_admin/owner (observability:read-content). Every other role sees the same trace with the bodies hidden — the metadata and optimizer decisions stay visible.

Spend and savings

The Home dashboard reads GET /admin/spend/dashboard. Its headline numbers:

KPI	What it means
Tokens saved	Input tokens the optimizer trimmed.
Total saving	Trimmed tokens as a share of addressable tokens — everything the optimizer could act on (sent fresh, not already served from the provider's cache).
Value created	The list-price value of what ran; its breakdown splits into what a subscription covers vs. what's billable.
Extra capacity	How much further the same subscription usage limits stretch (shown only when you have seats).

Track spend as cost per correct answer, not just raw dollars.

Each spend row carries a billing lane that sets how its cost and savings count:

Lane	Seat?	Cost & savings
api-key	No	Full cost, full savings.
subscription (included)	Yes	Configured over-allowance fraction.
subscription-extra (provider-confirmed)	Same seat	Full real cost and savings.

Lanes come from quota response metadata only — no key, prompt, or response body is stored. To check reported savings against real measurements, sample a shadow baseline or run an audited holdout.

Attribution across providers and models

Anyray fronts every provider, so one console attributes usage no matter which tool or model produced it — no stitching together a separate dashboard per vendor. Spend, savings, and tokens roll up three ways on GET /admin/spend/dashboard, each with a per-day trend:

Breakdown	What you see
Per user	Each developer's tokens, spend, and savings — with per-model, per-day detail on the Usage page.
Per team	The same rollup by team, from the x-anyray-metadata attribution.
Per model & provider	Tokens and savings per model, and every model carries its provider — so one view spans Claude, Codex, and every other provider the gateway routes to, side by side.

It's all metadata only — token counts and attribution, never prompt or response content.

Custom attribution dimensions

Beyond the fixed user/team/session set, a request can carry arbitrary named dimensions — per-feature, per-deployment, per-tenant — so you can slice spend by any axis without a gateway change. Send one header per dimension, modelled on Helicone's Helicone-Property-<Name>:

Custom attribution headers

x-anyray-property-feature: checkout
x-anyray-property-deployment: prod-eu-1

Each dimension is recorded on the spend row under attribution.properties and is groupable at GET /admin/spend/usage/properties?name=<dimension> (for example name=feature), which returns per-value token, spend, and savings totals.

The dimensions are content-free metadata, and the recording seam bounds them so a header can never become a content channel:

• Up to 16 custom dimensions per request; extras are dropped.
• Dimension names are lower-cased and must match [A-Za-z0-9][A-Za-z0-9_.-]{0,63} (≤ 64 characters); a name that fails the allowlist is skipped, never rejected.
• Dimension values are truncated at 1024 characters, matching the other attribution ids.
• A custom name that collides with a fixed dimension (e.g. cohort) is namespaced under properties and cannot overwrite the enforced value.

note

A malformed custom-attribution header is dropped, not fatal — it never fails the request, and the raw header value is never logged.

Usage and caps

The Usage page (GET /admin/spend/usage) also shows a live cap gauge — used / cap / percent — for capped users. Set caps on the Users page (or PUT /admin/user-caps).

When a capped user crosses a share of their cap (default 80% and 100%), the gateway posts one alert to a Slack-compatible webhook and logs a security event — once per user, threshold, and month across the whole fleet.

Subscription spend connectors

When first-party traffic bypasses the gateway, connectors can import metadata-only usage from Cursor, Devin, and GitHub Copilot into a separate local table. Imported rows do not affect gateway spend, token caps, Billing rollups, or savings. Values stay in each vendor's units.

Configure sources under Settings → Providers (the Subscription spend connectors section) or /admin/spend-connectors. Credentials are encrypted and write-only. See Subscription spend connectors for permissions, retention, GDPR behavior, and outbound hosts.

Traces

A trace records one request end-to-end, metadata only unless content mode keeps the bodies:

Row	What it captures
Trace	The request — session, user, status, cost, token totals.
Span	Each optimizer decision — kind, summary, tokens/cost saved.
Generation	Each provider attempt — model, usage, timing (one per retry).

Token counts include the cache split, and promptTokens is the full billed input (fresh + cache read + cache write), so cost stays correct for prompt-caching traffic like Claude Code and Codex. Full field list: Gateway → Spend and metering.

Traces are the only store that can hold content, and only when ANYRAY_CONTENT_MODE keeps it. They're pruned after 90 days by default (ANYRAY_SPEND_RETENTION_DAYS), and a GDPR delete erases a user's traces. Auth-event rows keep a reason code and 30 days of history for the Users → Diagnose panel.

Where it's stored

Everything derives from Postgres; the gateway is the only writer.

Store	Table(s)	Read by
Spend	anyray_spend	/admin/spend/dashboard · /admin/spend/usage
Trace	anyray_traces + anyray_observations	trace drill-down (/admin/observability/*)
Auth events	anyray_auth_events	/admin/auth-events