Deploy on Azure

Launch the whole stack on managed AKS from Azure Cloud Shell.

The script registers the resource providers, provisions the cluster, installs the bundled Helm chart (gateway, optimizer, console, Postgres), and exposes the gateway API (:8787) and console (:3000) as load balancers scoped to the CIDR you choose. This is the Azure analog of the AWS and GCP quick-launches; already run your own cluster? Use Kubernetes (Helm) instead.

Prerequisites

• An Azure subscription with billing enabled and permission to create AKS, Virtual Network, public IP, load balancer, and managed-disk resources.
• The Microsoft.ContainerService and Microsoft.Network resource providers — the deploy script registers them for you.
• A narrow ALLOWED_CIDR — your office or VPN range. For a single workstation: curl -fsS https://api.ipify.org, then <that-ip>/32.
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment) — required for usage metering.

Azure Cloud Shell already has az, kubectl, and helm — nothing to install locally.

Install

1

Open Azure Cloud Shell

Click to open Azure Cloud Shell (Bash), then clone the install repo in the next step.

Open Azure Cloud Shell

Prefer your own terminal?

Any machine with the Azure CLI (az), kubectl, and helm works — see Install from your own terminal below.

2

What you're deploying

A managed AKS cluster with two CIDR-scoped load balancers in front:

Service	Where	What it is
gateway	LoadBalancer :8787	OpenAI-compatible multi-provider API
proxy (console)	LoadBalancer :3000	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	ClusterIP (internal)	Request/response optimization hook
Postgres	ClusterIP + PVC (internal)	Spend + trace store — spend rows are metadata only; trace content per ANYRAY_CONTENT_MODE (encrypted default)

The Postgres and gateway/optimizer /data volumes bind AKS's default-annotated default StorageClass (Azure Disk CSI) — nothing to pre-create. Every secret (ANYRAY_ADMIN_TOKEN, ANYRAY_CONTENT_KEY, the pseudonym salt) is generated locally by setup.sh and stored as the anyray-secrets Kubernetes Secret — never printed.

3

Set your inputs

Clone the repo, export your two required values, then run the deploy script:

Cloud Shell

git clone https://github.com/anyrayHQ/install anyray && cd anyray

export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai
# Optional overrides:
# export LOCATION="eastus"
# export RESOURCE_GROUP="anyray"
# export CLUSTER="anyray"
# export NAMESPACE="anyray"
# export IMAGE_TAG="latest"              # or pin vX.Y.Z

./azure/deploy.sh

The script registers the resource providers, creates the resource group and AKS cluster, generates the secrets, installs the chart, waits for the load balancer IPs, and prints your URLs and admin key.

4

Wait for first boot

The AKS cluster is the long pole (~5–8 min), then the load balancers take ~1–3 min to get public IPs. The gateway restarts until Postgres is reachable, then goes healthy.

5

Get your URLs and admin key

deploy.sh prints the Console URL, Gateway URL, and admin key at the end. Read the admin key again any time:

kubectl get secret anyray-secrets -n anyray \
  -o jsonpath='{.data.ANYRAY_ADMIN_TOKEN}' | base64 --decode; echo

6

Open the console

Open the Console URL and sign in with that admin key, then follow After deployment below.

7

Verify the deployment

export GATEWAY_URL="http://your-gateway-lb-ip:8787"
export ADMIN_TOKEN="..."   # from the deploy output

curl -fsS "$GATEWAY_URL/" && echo "gateway ok"
# Deployment health — gateway / observability / spend / optimizer / portal:
curl -fsS "$GATEWAY_URL/admin/health" -H "Authorization: Bearer ${ADMIN_TOKEN}"

/admin/health returns 503 and names the failing leg if any required service is down; a healthy stack returns "ok": true with every leg green.

What runs where

Everything above runs in your Azure subscription. To point your local coding tools (Claude Code, Cursor, Codex) at the gateway, run npx anyray-connect --gateway <GatewayURL> on your laptop. See the developer FAQ.

Install from your own terminal

From any machine with the Azure CLI (az), kubectl, and helm authenticated to your subscription (az login):

git clone https://github.com/anyrayHQ/install anyray && cd anyray

export SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai

./azure/deploy.sh

The script is idempotent: re-running it reuses an existing cluster and keeps the existing anyray-secrets rather than rotating the content key. The inputs:

Variable	Required	Default	What it is
SUBSCRIPTION_ID	no	az's active subscription	Azure subscription to deploy into.
ALLOWED_CIDR	yes	—	CIDR allowed to reach the console/gateway LBs; 0.0.0.0/0 is rejected.
DEPLOYMENT_TOKEN	yes	—	Anyray Billing app deployment token (adt_…).
LOCATION	no	eastus	Azure region.
RESOURCE_GROUP	no	anyray	Resource group (created if missing, reused if present).
CLUSTER	no	anyray	AKS cluster name (reused if present).
NAMESPACE	no	anyray	Kubernetes namespace.
NODE_VM_SIZE	no	Standard_D2s_v5	Node pool VM size.
NODE_COUNT	no	2	Node pool size.
IMAGE_TAG	no	latest	Anyray image tag; pin vX.Y.Z for a reproducible deploy.
DEFAULT_MODEL	no	anthropic/claude-sonnet-4-5	Target for the anyray-default model alias.

Do not use 0.0.0.0/0

The console and gateway carry your org's spend data and admin access. deploy.sh rejects 0.0.0.0/0 — still choose the narrowest CIDR that works for your team.

After deployment — roll out to your users

Users reach the gateway at the Gateway URL from the deploy output.

1

Add a provider key

Provider keys stay server-side — callers never hold them. Add at least one from the console Providers page; multi-field providers (Bedrock, Vertex AI, Azure OpenAI) are configured there too. See Configure → Providers.

2

Send a test request

On the Providers page use Send test request, then confirm a row under Spend and a trace under Traces — that exercises routing, attribution, and the trace store end to end.

3

Enroll your users

/v1 is always gated — each user enrolls before their traffic is accepted. Mint an enrollment link (enl_…) on the console Users page; each user runs it on their own machine:

npx anyray-connect --enroll https://…/enroll/enl_…

That points their coding tools (Claude Code, Cursor, Codex, …) at the gateway and mints a personal key bound to their email — no provider keys ever touch a user's machine. For a whole fleet, push one provisioning token via your MDM — see Bulk enrollment with your MDM.

4

Track the migration and confirm the cutover

Watch the Users roster move from Pending to Enrolled; unenrolled /v1 traffic is always rejected (401). See Configure → Access control.

You're live

Your users' everyday AI tools now run through Anyray — spend attributed by user/team/model, content handled per your privacy mode, the optimizer cutting cost on the hot path.

Configuration

Gateway hardening and optimizer tuning are Helm values — set them in my-values.yaml or pass --set on a helm upgrade; the gateway rolls with the new value:

Helm value	What it controls
gateway.hsts	Emits HSTS headers (set only when serving over HTTPS).
gateway.contentMode / gateway.allowPlaintext	Content-privacy mode (encrypted by default) and the deploy gate for plaintext.
gateway.rateLimitRpm	Per-identity /v1 request limit per minute.
gateway.rateLimitIpRpm	Per-source-IP /v1 request limit per minute.
gateway.rateLimitUnauthRpm	Unauthenticated endpoint request limit per minute.
gateway.maxConcurrentRequests	Max simultaneous /v1 requests per identity or IP.
gateway.maxBodyBytes	Max request body size in bytes.
gateway.optimizerTimeoutMs	Normal optimizer timeout; defaults to 800.
gateway.optimizerVisionTimeoutMs	Vision optimizer timeout; defaults to 10000.

To re-scope the load balancers to a different CIDR later, update gateway.service.loadBalancerSourceRanges and proxy.service.loadBalancerSourceRanges and run helm upgrade. Other controls match Kubernetes (Helm).

Troubleshoot

A LoadBalancer Service has no external IP

AKS provisions the public IP in ~1–3 min:

kubectl get svc gateway anyray-proxy -n anyray

If EXTERNAL-IP stays <pending> for several minutes, confirm the subscription has public-IP quota in the region and that the cluster's managed identity has the Network Contributor role on the AKS-managed node resource group (MC_<resource-group>_<cluster>_<location>).

The console or gateway is unreachable from your network

The load balancers only admit traffic from ALLOWED_CIDR. Confirm your current public IP is inside that range (curl -fsS https://api.ipify.org), and re-scope if needed:

helm upgrade anyray ./helm -f my-values.yaml --namespace anyray --reuse-values \
  --set gateway.service.loadBalancerSourceRanges[0]=<your-cidr> \
  --set proxy.service.loadBalancerSourceRanges[0]=<your-cidr>

A pod is stuck Pending or CrashLooping

kubectl get pods -n anyray
kubectl describe pod -n anyray <pod>
kubectl logs -n anyray deployment/anyray-gateway

A Pending Postgres or *-data PVC usually means no default StorageClass — AKS ships a default-annotated default class (Azure Disk CSI); confirm with kubectl get storageclass. A gateway that keeps restarting is most often still waiting on Postgres.

Upgrade

IMAGE_TAG=latest follows the moving release channel. Re-run ./azure/deploy.sh to converge, or pin a version and roll the chart directly:

helm upgrade anyray ./helm -f my-values.yaml \
  --namespace anyray --reuse-values --set image.tag=vX.Y.Z

A pinned :vX.Y.Z keeps rollbacks reproducible and auditable; return to latest to follow the moving channel again.

Want to manage Helm yourself?

Already run a cluster, or want full control over namespaces, ingress, and storage classes? Use Kubernetes (Helm) — same chart, your cluster.