Choose your setup
Before you install
Every Anyray deployment needs a deployment token from the Anyray Billing app.
- Open app.anyray.ai and sign in with your work email.
- Mint the deployment token.
- Use that
adt_...token when you run the install command.
If your organization's network or security policy blocks the single sign-on redirect, use the "Sign in with email instead" link on the login page — it sends a one-time email code and skips the enterprise-IdP handoff, so you can still mint the deployment token.
Install Anyray
Anyray installs from published images. The whole stack (gateway, console, optimizer, and Postgres) comes up from
github.com/anyrayHQ/install.
Where will Anyray run?
Situation A — "I already have a gateway"
Recommended: put Anyray in front of your gateway. Anyray receives the requests, runs
its full pipeline (validate → attribute → optimizer → encrypt → meter → entitlement), then
forwards to your gateway as an OpenAI-compatible upstream — the anyray-upstream
provider. Every privacy and governance guarantee holds because Anyray stays on the path.
See the gateway for the configuration (custom_host,
ANYRAY_CUSTOM_HOST_ALLOWLIST).
Optional LiteLLM attach mode — only if an existing LiteLLM endpoint must stay user-facing. Attach mode runs just the Anyray optimizer + console, called from LiteLLM callbacks; gateway features (provider keys, routing, verified user connect links, the gateway spend store) are unavailable. See Attach to LiteLLM.
Situation B — "I call providers directly"
Simple providers — one API key. Set one environment variable on the gateway:
ANYRAY_PROVIDER_KEY_<SLUG>=<your-key>
<SLUG> is the provider slug, upper-cased, with non-alphanumerics collapsed to _
(azure-openai → AZURE_OPENAI) — the same convention for every supported provider. The
gateway uses this provider key after authenticating the caller's Anyray ark_… key. Without it, a
BYO client must send both credentials. You can also set keys on the console Providers page — see
Configure → Providers.
| Provider | Slug | Environment variable |
|---|---|---|
| OpenAI | openai | ANYRAY_PROVIDER_KEY_OPENAI=sk-... |
| Anthropic | anthropic | ANYRAY_PROVIDER_KEY_ANTHROPIC=sk-ant-... |
| Groq | groq | ANYRAY_PROVIDER_KEY_GROQ=gsk_... |
| Mistral AI | mistral-ai | ANYRAY_PROVIDER_KEY_MISTRAL_AI=... |
| DeepSeek | deepseek | ANYRAY_PROVIDER_KEY_DEEPSEEK=... |
| Cohere | cohere | ANYRAY_PROVIDER_KEY_COHERE=... |
google | ANYRAY_PROVIDER_KEY_GOOGLE=... |
Multi-credential providers — a credential "bag". Providers that need several fields
(a region, a secret, a service-account JSON, a resource endpoint) are configured as a
bag of named fields via the console Providers page or the admin API
(GET/PUT /admin/provider-keys, gated by ANYRAY_ADMIN_TOKEN). Field names are the
gateway's own option names, so a stored bag maps straight onto the outgoing request; the
bag stays server-side and is audit-logged by provider slug — never the values.
| Provider | Slug | Bag fields (typical) |
|---|---|---|
| AWS Bedrock | bedrock | awsRegion; uses the gateway AWS role by default (awsAccessKeyId, awsSecretAccessKey, awsSessionToken optional) |
| Google Vertex AI | vertex-ai | vertexProjectId, vertexRegion, vertexServiceAccountJson |
| Azure OpenAI | azure-openai | resourceName, azureDeploymentName, azureApiVersion, apiKey |
Storage and rotation details: Configure → Providers and Security → server-held provider keys.
OpenAI, Anthropic, Vertex AI, Azure, and Bedrock have dedicated pages under Integrations.