Deploy on a Compute Engine VM

Run the whole stack with docker compose on one Compute Engine VM.

The deploy script creates the VM with a persistent data disk, generates every secret on the box, and brings up the gateway, optimizer, console, and Postgres, with :3000 and :8787 firewall-scoped to the CIDR you choose. It's a VM rather than Cloud Run because the gateway keeps provider keys and user enrollments as files on a persistent /data volume: durable local disk, not an ephemeral serverless filesystem.

Prerequisites

• A Google Cloud project with billing enabled and permission to create Compute Engine instances, disks, and firewall rules. The script enables the Compute Engine and IAP APIs for you; IAP backs the SSH used to read your admin key, and no SSH port is exposed to the internet.
• A narrow ALLOWED_CIDR: your office or VPN range. For a single workstation: curl -fsS https://api.ipify.org, then <that-ip>/32. The script rejects 0.0.0.0/0; the console and gateway carry your org's spend data and admin access.
• Anyray deployment token (adt_…)

Cloud Shell already has gcloud; nothing to install locally.

Install

1

Open in Cloud Shell

Clones the install repo into Cloud Shell and opens the guided walkthrough.

Open in Cloud Shell

Any machine with the gcloud SDK works too; see Install from your own terminal below.

2

Set your inputs and run the script

Cloud Shell

export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai
# Optional overrides:
# export ZONE="us-central1-a"
# export INSTANCE="anyray"
# export MACHINE_TYPE="e2-standard-2"    # 2 vCPU / 8 GB
# export DISK_SIZE="200GB"         # holds the 90-day trace retention window
# export IMAGE_TAG="latest"              # or pin vX.Y.Z

./gcp/gce/deploy.sh

The script enables the APIs and creates the firewall rules and the VM. The VM's first-boot script installs Docker, generates the secrets on the box (ANYRAY_ADMIN_TOKEN, ANYRAY_CONTENT_KEY, the Postgres password, the pseudonym salt), connects to the Anyray Billing App, and starts the stack. All container state (the /data volumes and Postgres) lives on a dedicated persistent disk that is not deleted with the VM, so provider keys and enrollments survive a VM rebuild.

Service	Where	What it is
gateway	host :8787 (CIDR-scoped)	OpenAI-compatible multi-provider API
proxy (console)	host :3000 (CIDR-scoped)	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	in-VM only	Request/response optimization hook
Postgres	in-VM only	Spend + trace store

3

Wait for first boot

First boot installs Docker and pulls the images, so allow ~3–6 min. deploy.sh polls over IAP SSH and prints your URLs and admin key when the stack is up.

4

Open the console

From a machine inside your ALLOWED_CIDR, open the printed Console URL and sign in with the printed admin key. Read the key again any time over IAP SSH:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "sudo sed -n 's/^ANYRAY_ADMIN_TOKEN=//p' /opt/anyray/.env | head -n1"

5

Verify the deployment

Run the bundled per-leg check on the VM:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "cd /opt/anyray && sudo ./scripts/verify-deploy.sh"

It polls liveness, queries the admin-gated /admin/health, and prints a per-leg verdict (gateway / observability / spend / optimizer / portal) with a remedy for any failing leg.

To finish setup from your coding agent instead of the console, continue with Run Anyray from your AI agent.

Everything above runs in your Google Cloud project. Point your local coding tools at the gateway with npx anyray-connect@latest --gateway <GatewayURL> (developer FAQ).

Install from your own terminal

From any machine with the gcloud SDK authenticated to your project:

git clone https://github.com/anyrayHQ/install anyray && cd anyray

export PROJECT_ID="my-gcp-project"
export ALLOWED_CIDR="203.0.113.0/24"     # your office / VPN range
export DEPLOYMENT_TOKEN="adt_..."        # from app.anyray.ai

./gcp/gce/deploy.sh

The script is idempotent: re-running it reuses an existing VM (it never recreates the instance, so data and secrets are preserved) and leaves existing firewall rules in place. The inputs:

Variable	Required	Default	What it is
PROJECT_ID	yes	gcloud's active project	GCP project to deploy into.
ALLOWED_CIDR	yes	n/a	CIDR allowed to reach the console/gateway; 0.0.0.0/0 is rejected.
DEPLOYMENT_TOKEN	yes	n/a	Anyray Billing App deployment token (adt_…).
ZONE	no	us-central1-a	VM zone.
INSTANCE	no	anyray	VM name (reused if present).
MACHINE_TYPE	no	e2-standard-2	VM size (2 vCPU / 8 GB).
DISK_SIZE	no	200GB	Persistent data-disk size. Holds the gateway's 90-day trace retention window; measure your rate.
IMAGE_TAG	no	latest	Anyray image tag; pin vX.Y.Z for a reproducible deploy.
DEFAULT_MODEL	no	anthropic/claude-sonnet-5-5	Target for the anyray-default model alias.
NETWORK	no	default	VPC network for the VM + firewall rules.

Configuration

The stack is configured through /opt/anyray/.env on the VM. SSH in, edit, and re-apply; the knobs are the same as Local / VM:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap
cd /opt/anyray
sudo nano .env
sudo docker compose up -d        # recreate changed services

To re-scope the console/gateway to a different CIDR later:

gcloud compute firewall-rules update anyray-allow-web --source-ranges <your-cidr>

The VM's external IP is ephemeral; it can change if the VM is recreated. For a fixed address, reserve a static external IP, assign it to the VM, then re-run setup.sh --host <static-ip> on the box so the gateway reports the stable URL.

A full data disk stops Postgres and the spend and trace stores with it; DISK_SIZE defaults to 200GB, which holds the default 90-day trace window plus the images and /data volumes (measure your rate).

Troubleshoot

The deploy finished but printed no admin key

First boot was still running, or IAP SSH wasn't reachable yet:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "sudo tail -n 50 /var/log/anyray-startup.log"
gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap \
  --command "sudo sed -n 's/^ANYRAY_ADMIN_TOKEN=//p' /opt/anyray/.env | head -n1"

The console or gateway is unreachable from your network

The firewall only admits traffic from ALLOWED_CIDR. Confirm your current public IP is inside that range (curl -fsS https://api.ipify.org), and re-scope if needed:

gcloud compute firewall-rules update anyray-allow-web --source-ranges <your-cidr>

A container is unhealthy or restarting

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap
cd /opt/anyray
sudo docker compose ps
sudo docker compose logs --tail 100 gateway

A gateway that keeps restarting is most often still waiting on Postgres. Re-run the per-leg check: sudo ./scripts/verify-deploy.sh.

Upgrade

IMAGE_TAG=latest follows the moving release channel; soft updates (image-only releases) apply automatically by default (console: Settings → Updates), and the one-click Update now button covers the rest. To converge manually, or to pin a tag:

gcloud compute ssh anyray --zone us-central1-a --tunnel-through-iap
cd /opt/anyray
sudo git fetch && sudo git reset --keep origin/main
sudo ./gcp/gce/upgrade.sh
# Optional immutable-tag override: sudo ./gcp/gce/upgrade.sh vX.Y.Z

The git step refreshes the compose files and keeps your .env. upgrade.sh then pulls and restarts on that template. It also replaces the image pin first-time setup wrote to .env, which would otherwise override the refreshed template.

Outgrowing a single box? Google Cloud (GKE) runs the same stack on managed Kubernetes with rolling deploys.