Skip to main content

The data boundary

Anyray is fully self-hosted. The whole system — gateway, console, optimizer, the trace backend, and every datastore — runs from one root docker-compose inside your environment. Nothing leaves the box. There is no Anyray vendor, no SaaS, no billing API, no license check, and no egress of any kind — not even a content-free counter. Self-hosted is the boundary: it's air-gapped by construction.

Inside that boundary, two more guarantees protect your users' content:

  • Prompt/response content is encrypted at rest by default — no human ever reads it through the console.
  • The only metadata Anyray records is a content-free spend store — who/team, model, provider, tokens, cost, latency. Never content.

Everything lives inside the green box. No arrow crosses it — there is nowhere for your data to go. Content is encrypted at rest; the spend store is content-free.

What Anyray records (and what it doesn't)​

Anyray records exactly two things, both inside your environment:

  1. A content-free spend store. For every request: who and which team (from the x-anyray-metadata header), the model, the provider, token counts, cost, latency, and status — never the prompt or response. This powers Spend-by-user/team in the console and the admin-gated summary at GET /admin/spend.
  2. Per-request traces with content encrypted at rest. The gateway exports traces to a self-hosted trace backend. By default these are metadata-only; where content is stored it is encrypted with AES-256-GCM under ANYRAY_CONTENT_KEY. Humans see ciphertext in the console — decrypting requires the key and is an offline, authorized audit step, never something a UI exposes.

No prompts or responses are ever shown to a human, and nothing is ever sent off the box.

Content modes​

The org-wide content mode is set by ANYRAY_CONTENT_MODE and is changeable at runtime from the console Privacy settings page (every change is audit-logged):

  • encrypted (default) — content stored, encrypted at rest (AES-256-GCM).
  • off — store no content at all; only the content-free metadata is kept.
  • plaintext — content stored in the clear. This is deploy-gated: it only takes effect when the operator sets ANYRAY_ALLOW_PLAINTEXT=true at deploy time. The fail-safe never degrades up to plaintext on its own.

Why it's built this way​

  • Compliance scope stays small. Because no data ever leaves your environment, there is no third-party data processor to add to GDPR / SOC2 scope. (Not legal advice.)
  • Shorter vendor security review. "It's self-hosted, your data never leaves, and content is encrypted at rest" is a much easier review to pass.
  • No trust required. There's no egress to audit and no vendor to take on faith — the guarantee is structural, not a promise.

Learning stays inside the boundary too​

If you opt into adaptive optimization (roadmap), the optimizer learns from your traffic entirely on-prem — it reads only the local spend store and traces and writes only your local optimizer config. Nothing learned is ever sent anywhere, because nothing is ever sent anywhere. "Anyray gets better at your costs" stays fully compatible with "your data never leaves your environment."

See Org Admin → Security and Org Admin → Configure.