Usage governance

Anyray attributes every request to a user/team and can cap or exclude them.

Setting	Default	What it does
Per-user caps	none	Monthly token cap per user + near-cap alerts via GET / PUT /admin/user-caps.
Per-user budgets	none	USD spend budget per user: hard ceiling (402 at the limit), alert-only soft threshold, and a never / daily / weekly / monthly UTC reset window. Same config and endpoint as the caps above.
ANYRAY_METER_PER_USER	false	Also report per-user spend and savings.
ANYRAY_SEAT_EXCLUDE	unset	CSV of user ids / globs excluded from the billed seat count.
ANYRAY_SPEND_RETENTION_DAYS	spend: forever · traces: 90d	Auto-prune spend rows older than N days. Also sets the trace window unless one is set on the console Privacy page.

Budgets live on the console Users page; token caps go through PUT /admin/user-caps, and PATCH /admin/user-caps/:user adjusts one user (pair it with a scoped admin API key). A change reaches every replica within ~30 seconds. Both are soft, fail-open brakes: unattributed requests and everything already in flight pass through. Field reference: Spend & governance endpoints.

Rate limits

Set via .env + restart; an unset limit is off.

Env var	Default	Limit
ANYRAY_RATE_LIMIT_RPM	unset	Requests/min per key on /v1/*.
ANYRAY_RATE_LIMIT_IP_RPM	unset	Requests/min per source IP.
ANYRAY_RATE_LIMIT_UNAUTH_RPM	10	Requests/min per IP, unauthenticated.
ANYRAY_MAX_CONCURRENT_REQUESTS	unset	Max in-flight /v1/* per key.
ANYRAY_AUTH_FAIL_LIMIT	20	Counted credentialed auth failures per IP per 10 min; past it, further failing attempts get 429 (valid keys are never blocked).
ANYRAY_MAX_BODY_BYTES	33554432	Max request body (32 MiB); larger → 413.

How the failed-auth throttle counts, and what is always on: Abuse and DoS controls.

Measure datastore growth

Content is what grows; spend rows stay small (23 GB of traces vs 118 MB of spend on the same traffic). The default trace window is 90 days, and heavy whole-team plus CI-agent usage measured about 1 GB/day; measure your own rate in the first week. Both queries read byte lengths only, never content:

What the store holds now, and what it gains per day

-- Current footprint, largest tables first
SELECT relname AS table,
       pg_size_pretty(pg_total_relation_size(relid)) AS size
  FROM pg_catalog.pg_statio_user_tables
 ORDER BY pg_total_relation_size(relid) DESC
 LIMIT 5;

-- Content added per day over the last week (before compression)
SELECT date_trunc('day', start_time)::date AS day,
       pg_size_pretty(sum(octet_length(coalesce(input_enc, '')) +
                          octet_length(coalesce(output_enc, '')))) AS content
  FROM anyray_observations
 WHERE start_time > now() - interval '7 days'
 GROUP BY 1
 ORDER BY 1 DESC;

The daily figure is pre-compression (~2.5x smaller on disk); multiply by 90 for the steady state. Three levers cut the requirement:

Lever	Shortens	Notes
Trace retention (console Privacy page)	The trace and observation tables alone: the content, almost always what fills a volume	Spend history, savings reporting, and the billing period are unaffected.
ANYRAY_SPEND_RETENTION_DAYS	Spend rows, and traces when no trace window is set	Spend rows feed the console's history and the billing period. Anyray's own deployment runs 45.
ANYRAY_CONTENT_MODE=off (content mode)	Stores no content at all	Spend, savings, and every dashboard keep working; they were always metadata.