Skip to main content

Azure & others

Anyray isn't tied to any one cloud. The same model applies on Azure, on other Kubernetes distributions, and in the strictest isolated environments — only the redirect and credential mechanics differ. The Anyray gateway speaks Azure OpenAI natively.

The portable pattern​

  1. Run the whole stack in your environment (your cluster / account) — gateway, optimizer, console, and datastores. Anyray is fully self-hosted.
  2. Redirect endpoints by config — set the SDK base-URL env (OPENAI_BASE_URL / ANTHROPIC_BASE_URL) to the Anyray gateway (:8787) via the pod/Deployment spec, config management, or MDM. No org CA, no TLS-MITM, no HTTPS_PROXY. (Zero-touch admission-webhook injection is roadmap.)
  3. The gateway holds provider credentials/signing. The optimizer stays credential-free.
  4. Speak the same Optimizer Protocol v1 — nothing about the optimizer changes per cloud.

Azure​

  • Deploy on AKS (or your Azure VMs/containers) the same way as any Kubernetes target.
  • Set the SDK base-URL env to point pods at the Anyray gateway.
  • Azure OpenAI keys are held and used by the gateway, which speaks Azure OpenAI natively.

Other Kubernetes / VM environments​

Any environment that can run containers can run Anyray — the whole system comes up from one root docker compose up. The redirect is just an env var.

Strictly isolated / self-hosted environments​

Anyray is self-hosted by construction, so it's already air-gapped: there is no vendor egress at all — no billing aggregate, no SaaS, nothing leaves your environment. The content-free spend store and metadata-only traces stay local, and content is encrypted at rest by default (ANYRAY_CONTENT_MODE=encrypted). See The data boundary.

See also​