Configure

Provider keys, access, routing, spend caps, privacy mode, optimizer.

The console and the /admin/* API are the same settings behind admin auth (ANYRAY_ADMIN_TOKEN, or console SSO); every write is audit-logged.

Change via	When it applies	Covers
Console / /admin/*	Instantly	Provider keys, routing, content mode, optimizer, user caps
.env + restart	Next boot	ANYRAY_CONTENT_KEY, the plaintext gate, rate limits, timeouts

A fresh install works out of the box — usually you only need the first two or three steps.

1

Set a provider key

On Providers (or PUT /admin/provider-keys). Without one, BYO clients must send both the provider credential and their Anyray client key. See Provider keys.

2

Enroll users

/v1/* is always restricted to enrolled users — run anyray-connect --enroll / --sso first. See Access control.

3

Set a routing default

On Routing (or PUT /admin/routing-config). See Routing.

4

Pick a content mode

encrypted by default. See Content privacy.

5

(Optional) Tune the optimizer

A working seed ships. See Optimizer pipeline.

Provider keys

Set provider API keys on the Providers page (or GET / PUT /admin/provider-keys) instead of the ANYRAY_PROVIDER_KEY_* env vars — the runtime path wins. Keys stay server-side; changes are logged by provider, never the key value. See Security.

Access control

/v1/* is always restricted to enrolled users — no opt-out. Every request needs a valid client key, and that key's identity is authoritative for attribution (x-anyray-metadata can't override it). A missing, unknown, expired, or revoked key returns 401 (missing_key, key_unknown, key_expired, or key_revoked); with metering on, a deployment holding no billing lease returns 503 (entitlement_inactive). Enroll with anyray-connect --enroll or --sso before sending traffic.

Env var	Default	What it does
ANYRAY_VERIFIED_DEV_KEY_TTL_DAYS	90	Max lifetime of a minted client key (capped at the enrollment cert's expiry).

Rate limits

Set via .env + restart; an unset limit is off.

Env var	Default	Limit
ANYRAY_RATE_LIMIT_RPM	unset	Requests/min per key on /v1/*.
ANYRAY_RATE_LIMIT_IP_RPM	unset	Requests/min per source IP.
ANYRAY_RATE_LIMIT_UNAUTH_RPM	10	Requests/min per IP, unauthenticated.
ANYRAY_MAX_CONCURRENT_REQUESTS	unset	Max in-flight /v1/* per key.
ANYRAY_AUTH_FAIL_LIMIT	20	Counted credentialed auth failures per IP per 10 min; past it, further failing attempts get 429 (valid keys are never blocked).
ANYRAY_MAX_BODY_BYTES	33554432	Max request body (32 MiB); larger → 413.

SSRF allowlists and TLS headers → Security.

Routing

Which model or provider a request hits. Set the default on Routing (or GET / PUT /admin/routing-config); a client can override per request with the x-anyray-config header.

anyray-default — what anyray-connect sends to mean "gateway, you pick" — resolves to ANYRAY_DEFAULT_MODEL (default anthropic/claude-sonnet-4-5). Override per alias with ANYRAY_MODEL_ALIASES or PUT /admin/model-aliases. Routing applies to API-key traffic only — passthrough subscription traffic keeps the user's own token and provider.

Mode	Behavior
single	One target for every request.
loadbalance	Spread across targets.
fallback	Try targets in order; advance on failure.
conditional	Pick a target from request conditions.

Retry. Transient failures (502 / 503 / 504, network) get 2 attempts; a 429 is retried for non-streaming requests only. Add a retry block to change it — { "attempts": 0 } opts out, max 5.

Conditional routing picks a named target from an ordered conditions list, else default (no match and no default → 400):

{ "strategy": { "mode": "conditional",
    "conditions": [ { "query": { "metadata.team": "research" }, "then": "frontier" } ],
    "default": "economy" },
  "targets": [ { "name": "frontier", "provider": "anthropic" },
               { "name": "economy", "provider": "openai" } ] }

A query matches metadata.<key>, params.<field>, and url.pathname, with operators ($eq, $ne, $gt / $gte / $lt / $lte, $in, $nin, $regex) combined by $and / $or. First match wins.

Upstream timeouts bound the wait on a provider:

Env var	Default	Bounds
ANYRAY_UPSTREAM_CONNECT_TIMEOUT_MS	10000	Connecting to the provider.
ANYRAY_UPSTREAM_HEADERS_TIMEOUT_MS	60000	First response byte — raise for slow providers.
ANYRAY_UPSTREAM_BODY_TIMEOUT_MS	600000	Gap between streamed chunks.

Content privacy

ANYRAY_CONTENT_MODE sets how much request/response content is stored. The mode is runtime-mutable (Privacy page / PUT /admin/settings); the env var is only the boot default.

Mode	Behavior
encrypted (default)	Stored encrypted at rest (AES-256-GCM via ANYRAY_CONTENT_KEY) — humans see only ciphertext.
off	Store nothing — metadata only.
plaintext	Store readable content. Needs the gate ANYRAY_ALLOW_PLAINTEXT=true; debugging only.

The mode only ever degrades toward off: without the gate, plaintext falls back to encrypted or off, and encrypted refuses to boot without a valid ANYRAY_CONTENT_KEY (openssl rand -hex 32). Full semantics → Data boundary.

Plaintext debugging

Add the gate, restart, then select plaintext on the Privacy page. Remove the gate and switch back when done.

.env

ANYRAY_ALLOW_PLAINTEXT=true

Optimizer pipeline

The optimizer runs an ordered strategies[] array in optimizer.config.json (array order = run order). A working seed is baked into the image; first boot copies it to the data volume (ANYRAY_DATA_DIR). Override the path with ANYRAY_OPTIMIZER_CONFIG. Edit it on Settings or via GET / PUT /admin/optimizer/settings — the GET also lists every available strategy and its params. What each strategy does → Strategies and Guardrails.

The hook is wired by env: ANYRAY_OPTIMIZER_URL (unset = disabled; the gateway fails open) and ANYRAY_OPTIMIZER_TIMEOUT_MS (default 800 — past it, the request is forwarded un-optimized).

Per-target rules

overrides.rules runs a different pipeline per team, user, model, endpoint, or metadata.<key>. Each rule has a when matcher (globs, ANDed across dimensions, ORed within one) and an action — enable, disable, or params (disable wins):

{ "overrides": { "rules": [
  { "when": { "teams": ["platform"], "models": ["gpt-4o*"] }, "disable": ["prompt_compression"] }
] } }

Shadow baseline (estimate-vs-actual)

To audit reported savings, sample a shadow baseline: on the sampled fraction the gateway also measures the real trim next to the estimate (no extra upstream call). ANYRAY_OPTIMIZER_SHADOW_BASELINE (default false) — true measures all sampled requests, or a number up to 100 for that percentage. The drift is stored next to the estimate on the spend row.

Batch API input

The Batch API optimizes each line of a purpose=batch JSONL upload (POST /v1/files) when ANYRAY_OPTIMIZER_URL is set — fails open, uses retrieval-independent transformations, and permits a handle-free window_budget crop when the alternative is a provider context-limit rejection. Budget: ANYRAY_BATCH_OPTIMIZER_TIMEOUT_MS (default 30000); files over 100 MB skip optimization.

Usage governance

Anyray attributes every request to a user/team and can cap or exclude them:

Setting	Default	What it does
Per-user caps	none	Monthly token cap per user + near-cap alerts — GET / PUT /admin/user-caps.
ANYRAY_METER_PER_USER	false	Also report per-user spend and savings.
ANYRAY_SEAT_EXCLUDE	unset	CSV of user ids / globs excluded from the billed seat count.
ANYRAY_SPEND_RETENTION_DAYS	spend: forever · traces: 90d	Auto-prune spend rows older than N days.

Updates

Releases come in two shapes; your deployment tells them apart by itself:

• Soft — image-only: a pull and a container restart. The gateway confirms this locally against the release manifest (no new required settings, nothing flagged breaking).
• Hard — a new required env var, or a release the vendor flagged breaking — never applied automatically: the console shows Update needs an operator naming exactly what to set (names only — values never leave the deployment).

Platform	Soft updates
Docker / Compose	On this platform soft updates apply automatically by default — the gateway checks the release line and triggers the bundled updater, typically within ~15 minutes of a release.
Kubernetes / AWS / Railway	Never auto-applied — roll updates through the platform.

Toggle auto-apply in the console's sidebar Updates panel, or via the API (capability update:run):

PUT /admin/update/settings

{ "autoSoftUpdates": false }

Schema and data migrations

No manual migration step: on boot the gateway self-migrates its schema (once, serialized across replicas), and migrations are expand/contract-safe, so rolling updates and rollbacks keep working. Durable state — spend history, client keys, provider keys, caps, aliases — lives in Postgres, not the containers; nobody re-runs anyray-connect after an upgrade. To roll back, pin the previous ANYRAY_IMAGE_TAG and re-pull (latest records no prior version, so note yours before updating).

More configuration

These belong to install or operations, not this page:

• Security hardening — SSRF allowlists (ANYRAY_CUSTOM_HOST_ALLOWLIST, TRUSTED_CUSTOM_HOSTS), TLS headers → Security.
• Install infrastructure — public TLS, external Postgres (ANYRAY_SPEND_DB_URL), the mandatory Billing connection (setup.sh --connect) → install guide.
• GDPR export / erasure — /admin/gdpr/* → Compliance.
• Managed multi-tenancy (ANYRAY_MULTI_TENANT) — Anyray-hosted fleet mode only → Security.