Cloud providers overview
Anyray is designed to run inside your own cloud account, next to your workloads. Most of the deployment is the same everywhere; a few things — how endpoints get redirected and how provider credentials/signing work — are provider-specific.
What's the same on every cloud
- Everything runs in your account. Anyray is fully self-hosted — gateway, optimizer, console, and datastores all run next to your workloads. Nothing ever leaves; content is encrypted at rest by default.
- Config-based endpoint redirect. Workloads set their SDK base-URL env
(
OPENAI_BASE_URL/ANTHROPIC_BASE_URL) to point at the Anyray gateway (:8787) — via the pod/Deployment spec, config management, or a shell profile / CI secret. No org CA, no TLS-MITM, noHTTPS_PROXY. (Admission-webhook auto-injection is roadmap; lead with the explicit redirect.) - The gateway is multi-provider. Anyray's own gateway speaks OpenAI, Anthropic, Bedrock, Vertex, and Azure OpenAI natively, and calls the optimizer over Optimizer Protocol v1.
What's provider-specific
| Provider | Endpoint redirect | Credentials / signing | Page |
|---|---|---|---|
| GCP / GKE | base-URL env on workloads | gateway speaks Vertex natively (workload identity / attached SA) | GCP / GKE |
| AWS / Bedrock | base-URL env on workloads | gateway speaks Bedrock natively and handles SigV4 signing | AWS / Bedrock |
| Azure & others | base-URL env / config | gateway holds the provider keys (Azure OpenAI, etc.) | Azure & others |
A worked pattern
For Claude Code on Vertex, the Anyray gateway now speaks Vertex/Anthropic natively — so you can point Claude Code straight at it. A LiteLLM + Claude-on-Vertex stack remains a valid alternative pattern; see Vertex + Claude Code and GCP / GKE.
Reminder: the gateway owns provider credentials
The Anyray gateway holds provider keys and does request signing (including Bedrock SigV4); the optimizer only decides request transforms and never holds your provider credentials. See AWS / Bedrock.