Usage governance

Anyray attributes every request to a user/team and can cap or exclude them.

Setting	Default	What it does
Per-user caps	none	Monthly token cap per user + near-cap alerts. Set the notification destination on the console Health page; the config is GET / PUT /admin/user-caps.
Per-user budgets	none	USD spend budget per user: hard ceiling (402 at the limit), alert-only soft threshold, and a never / daily / weekly / monthly UTC reset window. Same config and endpoint as the caps above.
Subscription fallback	none	What serves a person when their own subscription is throttled (below). Replays bill at full API rates, so nothing happens until you set one.
ANYRAY_METER_PER_USER	false	Also report per-user spend and savings under the raw user id. Per-user prompt cache figures are sent either way, under a salted pseudonym (fields).
ANYRAY_SEAT_EXCLUDE	unset	CSV of user ids / globs excluded from the billed seat count.
ANYRAY_SPEND_RETENTION_DAYS	spend: forever · traces: 90d	Auto-prune spend rows older than N days. Also sets the trace window unless one is set on the console Privacy page.

Budgets live on the console Users page; token caps go through PUT /admin/user-caps, and PATCH /admin/user-caps/:user adjusts one user (pair it with a scoped admin API key). A change reaches every replica within about a second. Both are soft, fail-open brakes: unattributed requests and everything already in flight pass through. A request that carries no client key (identityMode: network-trusted) and names its user in x-anyray-metadata never fills that user's cap or budget, since anyone can name a colleague there. Field reference: Spend & governance endpoints.

Subscription fallback

A developer's own subscription seat serves their requests and costs the org nothing per token. When the seat hits its usage limit, that developer is blocked until the limit resets.

A fallback policy replays a throttled seat's requests on a server-held Anthropic API key instead. Replays bill as subscription-extra at full API rates, so nothing falls back until you set a policy.

Set it through the admin API; the console has no editor for it. Use an admin API key with policies:read and policies:write. This makes the stored Anthropic key ci-prod the fallback for everyone:

Set the org fallback

curl -fsS https://<your-anyray-gateway>/admin/v1/policies/teams \
  -H "Authorization: Bearer $ADMIN_API_KEY" |
jq '{expectedRevision: .revision, skills,
  fallback: (.fallback + {org: {mode: "fallback",
    targets: [{provider: "anthropic", providerKeyId: "ci-prod"}]}})}' |
curl -sS -X PUT https://<your-anyray-gateway>/admin/v1/policies/teams \
  -H "Authorization: Bearer $ADMIN_API_KEY" \
  -H "Content-Type: application/json" -d @-

A PUT (legacy PUT /admin/team-skills) replaces the whole skill set and the whole fallback, so the command sends back the current skills and merges into the stored fallback. A PUT without fallback leaves it alone; "fallback": {} clears it.

mode	What it does
fallback	Replays on up to 4 targets, in order, using the first whose key resolves. hourlyMaxRequests caps replays per person per hour (default 60).
off	No fallback. They stay blocked until their own limit resets, even when a broader scope has one.

Put a policy under org, teams.<team>, or users.<user>. User beats team beats org, and a scope with no entry follows the broader one.

Only an Anthropic API key can take over a seat ("provider": "anthropic"). The replay resends the same request with a different key, so Bedrock and Vertex cannot serve it. Leave out providerKeyId to use the configured Anthropic key. A save naming another provider, or a providerKeyId that is not stored, is refused.

Rate limits

Set via .env + restart; an unset limit is off.

Env var	Default	Limit
ANYRAY_RATE_LIMIT_RPM	unset	Requests/min per key on /v1/*.
ANYRAY_RATE_LIMIT_IP_RPM	unset	Requests/min per source IP.
ANYRAY_RATE_LIMIT_UNAUTH_RPM	10	Requests/min per IP, unauthenticated.
ANYRAY_MAX_CONCURRENT_REQUESTS	unset	Max in-flight /v1/* per key.
ANYRAY_AUTH_FAIL_LIMIT	20	Counted credentialed auth failures per IP per 10 min; past it, further failing attempts get 429 (valid keys are never blocked).
ANYRAY_MAX_BODY_BYTES	33554432	Max request body (32 MiB); larger → 413.

How the failed-auth throttle counts, and what is always on: Abuse and DoS controls.

Measure datastore growth

Content is what grows; spend rows stay small (23 GB of traces vs 118 MB of spend on the same traffic). The default trace window is 90 days, and heavy whole-team plus CI-agent usage measured about 1 GB/day; measure your own rate in the first week. Both queries read byte lengths only, never content:

What the store holds now, and what it gains per day

-- Current footprint, largest tables first
SELECT relname AS table,
       pg_size_pretty(pg_total_relation_size(relid)) AS size
  FROM pg_catalog.pg_statio_user_tables
 ORDER BY pg_total_relation_size(relid) DESC
 LIMIT 5;

-- Content added per day over the last week (before compression)
SELECT date_trunc('day', start_time)::date AS day,
       pg_size_pretty(sum(octet_length(coalesce(input_enc, '')) +
                          octet_length(coalesce(output_enc, '')))) AS content
  FROM anyray_observations
 WHERE start_time > now() - interval '7 days'
 GROUP BY 1
 ORDER BY 1 DESC;

The daily figure is pre-compression (~2.5x smaller on disk); multiply by 90 for the steady state. Three levers cut the requirement:

Lever	Shortens	Notes
Trace retention (console Privacy page)	The trace and observation tables alone: the content, almost always what fills a volume	Spend history, savings reporting, and the billing period are unaffected.
ANYRAY_SPEND_RETENTION_DAYS	Spend rows, and traces when no trace window is set	Spend rows feed the console's history and the billing period. Anyray's own deployment runs 45.
ANYRAY_CONTENT_MODE=off (content mode)	Stores no content at all	Spend, savings, and every dashboard keep working; they were always metadata.

Then tell Anyray the volume size so it can warn before the database fills it. On the console Privacy page, set Database storage in GiB, using the size the volume can grow to (a PVC's size or a storage-autoscaling maximum). Postgres cannot report its own disk size, so the warning measures against this number.