Security
Anyray's security posture rests on three pillars: everything runs in your own environment, prompt and response content is encrypted at rest and never shown to humans, and traffic is redirected without an org CA or TLS-MITM.
1. Everything runs in your environment
The whole system — the gateway, the optimizer, the console, the Langfuse-derived trace
backend, and every datastore — runs inside your own environment from one
docker compose up. Nothing egresses: there is no vendor, no usage aggregate, no
license check. Read the full model in The data boundary.
2. Content is encrypted at rest — humans see ciphertext
The data flowing through Anyray is your employees' own work (source code, internal docs,
proprietary data), so prompt and response content is encrypted at rest by default
(AES-256-GCM, key in ANYRAY_CONTENT_KEY). Humans never see it — the console and traces
are metadata-only; decrypting requires the key and is an offline, authorized-audit
action, never a UI feature.
Org-wide content modes (ANYRAY_CONTENT_MODE, changeable at runtime from the console):
| Mode | Behavior |
|---|---|
encrypted (default) | Content stored as ciphertext (AES-256-GCM). |
off | No content stored at all — nothing to decrypt. |
plaintext | Raw content — deploy-gated behind ANYRAY_ALLOW_PLAINTEXT=true; a console toggle alone can't expose it. |
The mode fails safe — it never silently degrades up to plaintext, and every change is audit-logged. See Configure and The data boundary.
3. One admin key gates the whole console
The entire console and every /admin/* API (spend, optimizer settings, content-privacy)
sit behind a single admin key (ANYRAY_ADMIN_TOKEN). One key, one gate — no per-page
logins, no separate Langfuse sign-in for day-to-day use.
4. Interception is config-based — no CA, no TLS-MITM
Workers reach Anyray by pointing their SDK base URL at the gateway — not by intercepting TLS:
- set
OPENAI_BASE_URL/ANTHROPIC_BASE_URLto the Anyray gateway (:8787) via the pod/Deployment spec, config management, a shell profile, or a CI secret
There is no org CA, no TLS-MITM, and no HTTPS_PROXY — the SDK simply talks to a
different base URL.
:::info Roadmap: zero-touch injection & Bedrock IRSA re-signing Auto-injecting the base-URL env (an in-cluster admission webhook, managed settings / MDM) and re-signing Bedrock upstream with the proxy's IRSA role are roadmap. Today the gateway speaks Bedrock/Vertex/Azure natively (it holds the provider credentials), and the redirect is the explicit, auditable base-URL override above. :::
Why this matters for your security review
- Nothing on the worker trusts a new certificate authority.
- No man-in-the-middle on TLS; the SDK just talks to a different base URL.
- The change is reversible and visible (it's an env var), not a system-wide trust change.
Secrets and provider credentials
The gateway holds provider API keys and performs provider signing (including Bedrock SigV4 and Vertex/Azure auth). The optimizer is credential-free — it only transforms the request and never takes custody of provider keys or content.
Server-held provider keys
Provider keys live server-side, never on the client. Workers and SDKs send only a placeholder key; the real provider credential stays in the gateway and is never returned to a caller. There are two ways to supply it:
- Env vars —
ANYRAY_PROVIDER_KEY_*, set at deploy time. - Runtime admin path — the console Providers page (or
GET/PUT /admin/provider-keys, behind the singleANYRAY_ADMIN_TOKEN), which lets you rotate keys without a redeploy. The runtime admin path takes precedence over the env vars.
Either way the key is stored server-side and never exposed to clients. Provider-key changes are audit-logged by provider slug only — never the key value — so the audit trail itself never leaks a secret. See Configure → Providers.
What a buyer's security team should walk away with
- Anyray runs entirely in your account; your prompts and responses never reach a vendor.
- Content is encrypted at rest and never shown to humans; the spend ledger is content-free.
- Nothing egresses — there is no outbound data at all.
- No new CA, no TLS interception, no proxy trust to manage.
- One admin key gates the whole surface.