Deploy on a Compute Engine VM

Run the whole stack with docker compose on one Compute Engine VM.

The deploy script creates the VM with a persistent data disk, generates every secret on the box, and brings up the gateway, optimizer, console, and Postgres, with :3000 and :8787 firewall-scoped to the CIDR you choose. It's a VM rather than Cloud Run because the gateway keeps provider keys and user enrollments as files on a persistent /data volume: durable local disk, not an ephemeral serverless filesystem.

Prerequisites

• A Google Cloud project with billing enabled and permission to create Compute Engine instances, disks, and firewall rules. The script enables the Compute Engine and IAP APIs for you; IAP backs the SSH used to read your admin key, and no SSH port is exposed to the internet.
• A narrow ALLOWED_CIDR: your office or VPN range. For a single workstation: curl -fsS https://api.ipify.org, then <that-ip>/32. The script rejects 0.0.0.0/0; the console and gateway carry your org's spend data and admin access.
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment).

Cloud Shell already has gcloud; nothing to install locally.

Install

1

Open in Cloud Shell

Clones the install repo into Cloud Shell and opens the guided walkthrough.

Open in Cloud Shell

Any machine with the gcloud SDK works too; see Install from your own terminal below.

2

Set your inputs and run the script

Cloud Shell

export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai
# Optional overrides:
# export ZONE="us-central1-a"
# export INSTANCE="anyray"
# export MACHINE_TYPE="e2-standard-2"    # 2 vCPU / 8 GB
# export DISK_SIZE="50GB"          # holds the 90-day trace retention window
# export IMAGE_TAG="latest"              # or pin vX.Y.Z

./gcp/gce/deploy.sh

The script enables the APIs and creates the firewall rules and the VM. The VM's first-boot script installs Docker, generates the secrets on the box (ANYRAY_ADMIN_TOKEN, ANYRAY_CONTENT_KEY, the Postgres password, the pseudonym salt), connects to Anyray Billing app, and starts the stack. All container state (the /data volumes and Postgres) lives on a dedicated persistent disk that is not deleted with the VM, so provider keys and enrollments survive a VM rebuild.

Service	Where	What it is
gateway	host :8787 (CIDR-scoped)	OpenAI-compatible multi-provider API
proxy (console)	host :3000 (CIDR-scoped)	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	in-VM only	Request/response optimization hook
Postgres	in-VM only	Spend + trace store

3

Wait for first boot

First boot installs Docker and pulls the images, so allow ~3–6 min. deploy.sh polls over IAP SSH and prints your URLs and admin key when the stack is up.

4

Open the console

From a machine inside your ALLOWED_CIDR, open the printed Console URL and sign in with the printed admin key. Read the key again any time over IAP SSH:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "sudo sed -n 's/^ANYRAY_ADMIN_TOKEN=//p' /opt/anyray/.env | head -n1"

5

Verify the deployment

Run the bundled per-leg check on the VM:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "cd /opt/anyray && sudo ./scripts/verify-deploy.sh"

It polls liveness, queries the admin-gated /admin/health, and prints a per-leg verdict (gateway / observability / spend / optimizer / portal) with a remedy for any failing leg.

Everything above runs in your Google Cloud project. Point your local coding tools at the gateway with npx anyray-connect@latest --gateway <GatewayURL> (developer FAQ).

Install from your own terminal

From any machine with the gcloud SDK authenticated to your project:

git clone https://github.com/anyrayHQ/install anyray && cd anyray

export PROJECT_ID="my-gcp-project"
export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai

./gcp/gce/deploy.sh

The script is idempotent: re-running it reuses an existing VM (it never recreates the instance, so data and secrets are preserved) and leaves existing firewall rules in place. The inputs:

Variable	Required	Default	What it is
PROJECT_ID	yes	gcloud's active project	GCP project to deploy into.
ALLOWED_CIDR	yes	n/a	CIDR allowed to reach the console/gateway; 0.0.0.0/0 is rejected.
DEPLOYMENT_TOKEN	yes	n/a	Anyray Billing app deployment token (adt_…).
ZONE	no	us-central1-a	VM zone.
INSTANCE	no	anyray	VM name (reused if present).
MACHINE_TYPE	no	e2-standard-2	VM size (2 vCPU / 8 GB).
DISK_SIZE	no	50GB	Persistent data-disk size. Holds the gateway's 90-day trace retention window; measure your rate.
IMAGE_TAG	no	latest	Anyray image tag; pin vX.Y.Z for a reproducible deploy.
DEFAULT_MODEL	no	anthropic/claude-sonnet-4-5	Target for the anyray-default model alias.
NETWORK	no	default	VPC network for the VM + firewall rules.

Configuration

The stack is configured through /opt/anyray/.env on the VM. SSH in, edit, and re-apply; the knobs are the same as Local / VM:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap
cd /opt/anyray
sudo nano .env
sudo docker compose up -d        # recreate changed services

To re-scope the console/gateway to a different CIDR later:

gcloud compute firewall-rules update anyray-allow-web --source-ranges <your-cidr>

The VM's external IP is ephemeral; it can change if the VM is recreated. For a fixed address, reserve a static external IP, assign it to the VM, then re-run setup.sh --host <static-ip> on the box so the gateway reports the stable URL.

A full data disk stops Postgres and the spend and trace stores with it; DISK_SIZE defaults to 50GB, which holds the default 90-day trace window for a typical team (measure your rate).

Troubleshoot

The deploy finished but printed no admin key

First boot was still running, or IAP SSH wasn't reachable yet:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "sudo tail -n 50 /var/log/anyray-startup.log"
gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "sudo sed -n 's/^ANYRAY_ADMIN_TOKEN=//p' /opt/anyray/.env | head -n1"

The console or gateway is unreachable from your network

The firewall only admits traffic from ALLOWED_CIDR. Confirm your current public IP is inside that range (curl -fsS https://api.ipify.org), and re-scope if needed:

gcloud compute firewall-rules update anyray-allow-web --source-ranges <your-cidr>

A container is unhealthy or restarting

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap
cd /opt/anyray
sudo docker compose ps
sudo docker compose logs --tail 100 gateway

A gateway that keeps restarting is most often still waiting on Postgres. Re-run the per-leg check: sudo ./scripts/verify-deploy.sh.

Upgrade

IMAGE_TAG=latest follows the moving release channel; soft updates (image-only releases) apply automatically by default (console: Settings → Updates), and the one-click Update now button covers the rest. To converge manually, or to pin a tag:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap
cd /opt/anyray
# (optional) pin a tag: set ANYRAY_IMAGE_TAG=vX.Y.Z in .env
sudo docker compose pull && sudo docker compose up -d

Outgrowing a single box? Google Cloud (GKE) runs the same stack on managed Kubernetes with rolling deploys.