Deploy on Google Cloud

Launch the whole stack on GKE Autopilot with one Cloud Shell click.

The script provisions the cluster, installs the bundled Helm chart (gateway, optimizer, console, Postgres), and exposes the gateway API (:8787) and console (:3000) as load balancers scoped to the CIDR you choose. This is the Google Cloud analog of the AWS quick-launch; already run your own cluster? Use Kubernetes (Helm) instead.

Prerequisites

• A Google Cloud project with billing enabled and permission to create GKE, Compute Engine, and load balancer resources.
• The Kubernetes Engine and Compute Engine APIs — the deploy script enables them for you.
• A narrow ALLOWED_CIDR — your office or VPN range. For a single workstation: curl -fsS https://api.ipify.org, then <that-ip>/32.
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment) — required for usage metering.

Cloud Shell already has gcloud, kubectl, and helm — nothing to install locally.

Install

1

Open in Cloud Shell

Click to clone the install repo into Cloud Shell and open the guided walkthrough.

Open in Cloud Shell

Prefer your own terminal?

Any machine with the gcloud SDK, kubectl, and helm works — see Install from your own terminal below.

2

What you're deploying

A GKE Autopilot cluster with two CIDR-scoped load balancers in front:

Service	Where	What it is
gateway	LoadBalancer :8787	OpenAI-compatible multi-provider API
proxy (console)	LoadBalancer :3000	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	ClusterIP (internal)	Request/response optimization hook
Postgres	ClusterIP + PVC (internal)	Spend + trace store — spend rows are metadata only; trace content per ANYRAY_CONTENT_MODE (encrypted default)

The Postgres and gateway/optimizer /data volumes bind Autopilot's default standard-rwo StorageClass — nothing to pre-create. Every secret (ANYRAY_ADMIN_TOKEN, ANYRAY_CONTENT_KEY, the pseudonym salt) is generated locally by setup.sh and stored as the anyray-secrets Kubernetes Secret — never printed.

3

Set your inputs

Export your two required values, then run the deploy script:

Cloud Shell

export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai
# Optional overrides:
# export REGION="us-central1"
# export CLUSTER="anyray"
# export NAMESPACE="anyray"
# export IMAGE_TAG="latest"              # or pin vX.Y.Z

./gcp/deploy.sh

The script enables the APIs, creates the Autopilot cluster, generates the secrets, installs the chart, waits for the load balancer IPs, and prints your URLs and admin key.

4

Wait for first boot

The Autopilot cluster is the long pole (~5–8 min), then the load balancers take ~1–3 min to get external IPs. The gateway restarts until Postgres is reachable, then goes healthy.

5

Get your URLs and admin key

deploy.sh prints the Console URL, Gateway URL, and admin key at the end. Read the admin key again any time:

kubectl get secret anyray-secrets -n anyray \
  -o jsonpath='{.data.ANYRAY_ADMIN_TOKEN}' | base64 --decode; echo

6

Open the console

Open the Console URL and sign in with that admin key, then follow After deployment below.

7

Verify the deployment

export GATEWAY_URL="http://your-gateway-lb-ip:8787"
export ADMIN_TOKEN="..."   # from the deploy output

curl -fsS "$GATEWAY_URL/" && echo "gateway ok"
# Deployment health — gateway / observability / spend / optimizer / portal:
curl -fsS "$GATEWAY_URL/admin/health" -H "Authorization: Bearer ${ADMIN_TOKEN}"

/admin/health returns 503 and names the failing leg if any required service is down; a healthy stack returns "ok": true with every leg green.

What runs where

Everything above runs in your Google Cloud project. To point your local coding tools (Claude Code, Cursor, Codex) at the gateway, run npx anyray-connect --gateway <GatewayURL> on your laptop. See the developer FAQ.

Install from your own terminal

From any machine with the gcloud SDK, kubectl, and helm authenticated to your project:

git clone https://github.com/anyrayHQ/install anyray && cd anyray

export PROJECT_ID="my-gcp-project"
export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai

./gcp/deploy.sh

The script is idempotent: re-running it reuses an existing cluster and keeps the existing anyray-secrets rather than rotating the content key. The inputs:

Variable	Required	Default	What it is
PROJECT_ID	yes	gcloud's active project	GCP project to deploy into.
ALLOWED_CIDR	yes	—	CIDR allowed to reach the console/gateway LBs; 0.0.0.0/0 is rejected.
DEPLOYMENT_TOKEN	yes	—	Anyray Billing app deployment token (adt_…).
REGION	no	us-central1	Autopilot region.
CLUSTER	no	anyray	Cluster name (reused if present).
NAMESPACE	no	anyray	Kubernetes namespace.
IMAGE_TAG	no	latest	Anyray image tag; pin vX.Y.Z for a reproducible deploy.
DEFAULT_MODEL	no	anthropic/claude-sonnet-4-5	Target for the anyray-default model alias.

Do not use 0.0.0.0/0

The console and gateway carry your org's spend data and admin access. deploy.sh rejects 0.0.0.0/0 — still choose the narrowest CIDR that works for your team.

After deployment — roll out to your users

Users reach the gateway at the Gateway URL from the deploy output.

1

Add a provider key

Provider keys stay server-side — callers never hold them. Add at least one from the console Providers page; multi-field providers (Bedrock, Vertex AI, Azure) are configured there too. See Configure → Providers.

2

Send a test request

On the Providers page use Send test request, then confirm a row under Spend and a trace under Traces — that exercises routing, attribution, and the trace store end to end.

3

Enroll your users

/v1 is always gated — each user enrolls before their traffic is accepted. Mint an enrollment link (enl_…) on the console Users page; each user runs it on their own machine:

npx anyray-connect --enroll https://…/enroll/enl_…

That points their coding tools (Claude Code, Cursor, Codex, …) at the gateway and mints a personal key bound to their email — no provider keys ever touch a user's machine. For a whole fleet, push one provisioning token via your MDM — see Bulk enrollment with your MDM.

4

Track the migration and confirm the cutover

Watch the Users roster move from Pending to Enrolled; unenrolled /v1 traffic is always rejected (401). See Configure → Access control.

You're live

Your users' everyday AI tools now run through Anyray — spend attributed by user/team/model, content handled per your privacy mode, the optimizer cutting cost on the hot path.

Configuration

Gateway hardening and optimizer tuning are Helm values — set them in my-values.yaml or pass --set on a helm upgrade; the gateway rolls with the new value:

Helm value	What it controls
gateway.hsts	Emits HSTS headers (set only when serving over HTTPS).
gateway.contentMode / gateway.allowPlaintext	Content-privacy mode (encrypted by default) and the deploy gate for plaintext.
gateway.rateLimitRpm	Per-identity /v1 request limit per minute.
gateway.rateLimitIpRpm	Per-source-IP /v1 request limit per minute.
gateway.rateLimitUnauthRpm	Unauthenticated endpoint request limit per minute.
gateway.maxConcurrentRequests	Max simultaneous /v1 requests per identity or IP.
gateway.maxBodyBytes	Max request body size in bytes.
gateway.optimizerTimeoutMs	Normal optimizer timeout; defaults to 800.
gateway.optimizerVisionTimeoutMs	Vision optimizer timeout; defaults to 10000.

To re-scope the load balancers to a different CIDR later, update gateway.service.loadBalancerSourceRanges and proxy.service.loadBalancerSourceRanges and run helm upgrade. Other controls match Kubernetes (Helm).

Troubleshoot

A LoadBalancer Service has no external IP

GKE provisions the external IP in ~1–3 min:

kubectl get svc gateway anyray-proxy -n anyray

If EXTERNAL-IP stays <pending> for several minutes, confirm the Compute Engine API is enabled and the project has quota for external IP addresses.

The console or gateway is unreachable from your network

The load balancers only admit traffic from ALLOWED_CIDR. Confirm your current public IP is inside that range (curl -fsS https://api.ipify.org), and re-scope if needed:

helm upgrade anyray ./helm -f my-values.yaml --namespace anyray --reuse-values \
  --set gateway.service.loadBalancerSourceRanges[0]=<your-cidr> \
  --set proxy.service.loadBalancerSourceRanges[0]=<your-cidr>

A pod is stuck Pending or CrashLooping

kubectl get pods -n anyray
kubectl describe pod -n anyray <pod>
kubectl logs -n anyray deployment/anyray-gateway

A Pending Postgres or *-data PVC usually means no default StorageClass — Autopilot ships standard-rwo; confirm with kubectl get storageclass. A gateway that keeps restarting is most often still waiting on Postgres.

Upgrade

IMAGE_TAG=latest follows the moving release channel. Re-run ./gcp/deploy.sh to converge, or pin a version and roll the chart directly:

helm upgrade anyray ./helm -f my-values.yaml \
  --namespace anyray --reuse-values --set image.tag=vX.Y.Z

A pinned :vX.Y.Z keeps rollbacks reproducible and auditable; return to latest to follow the moving channel again.

Want to manage Helm yourself?

Already run a cluster, or want full control over namespaces, ingress, and storage classes? Use Kubernetes (Helm) — same chart, your cluster.