Deploy on a remote VM over SSH

Deploy Anyray on a remote VM from your workstation over SSH.

Prerequisites

Remote host

• Docker Engine (v24+) running and reachable over SSH — only the daemon lives here; the docker compose CLI runs on your workstation and drives it over the SSH tunnel
• ~2 GB RAM free (the stack's real footprint is ~1.5 GB) — instance types per cloud under System requirements
• Ports 3000 and 8787 open to your org network / VPN CIDR only — never 0.0.0.0/0

Local workstation

• git, openssl, and Docker with the Compose v2 plugin on PATH — setup.sh and the compose CLI run here, not on the remote
• SSH key-based access to the remote host (ssh user@<vm> works without a password prompt)
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment) — required for usage metering

Install

On your local workstation:

1

Clone the install repo

git clone https://github.com/anyrayHQ/install anyray && cd anyray

2

Generate secrets and `.env`

./setup.sh --connect <adt_token> --host <vm-ip-or-hostname>

--host is how your org reaches the console and gateway; --connect wires the deployment to Anyray Billing app for metering. Secrets and .env are written locally.

3

Deploy over the SSH tunnel

DOCKER_HOST=ssh://user@<vm> docker compose up -d

Images are pulled and started on the remote host — gateway on :8787, console on :3000 of that host. Your .env stays on the workstation. Services:

Service	Exposed port	What it is
gateway	:8787	OpenAI-compatible multi-provider API
proxy (console)	:3000	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	internal	Request/response optimization hook
postgres	internal	Spend + trace store — spend rows are metadata only; trace content per ANYRAY_CONTENT_MODE (encrypted default)

4

Verify the install is healthy

DOCKER_HOST=ssh://user@<vm> docker compose ps
curl -fs http://<host>:8787/ && echo "gateway ok"

Expected: all services running; gateway and optimizer (healthy).

5

Open the console

Open http://<host>:3000 and sign in with the admin key setup.sh printed (also in your local .env as ANYRAY_ADMIN_TOKEN). Then follow After deployment below.

DOCKER_HOST vs SSH config

With a Host anyray-vm entry in ~/.ssh/config, DOCKER_HOST=ssh://anyray-vm works too.

After deployment — roll out to your users

The stack runs on the remote VM, but you drive it from your local .env; users connect over the VM's public URL.

1

Add a provider key

Provider keys stay server-side — callers never hold them. Add at least one on the console Providers page, or in your local .env, then re-apply:

.env (local)

ANYRAY_PROVIDER_KEY_OPENAI=sk-...        # or ANYRAY_PROVIDER_KEY_ANTHROPIC=sk-ant-...

DOCKER_HOST=ssh://user@<vm> docker compose up -d

The console path wins over the env var; multi-field providers (Bedrock, Vertex, Azure) are configured on the Providers page. See Configure → Providers.

2

Send a test request

On the Providers page use Send test request, then confirm a row under Spend and a trace under Traces — that exercises routing, attribution, and the trace store end to end.

3

Enroll your users

/v1 is always gated — each user enrolls before their traffic is accepted. Mint an enrollment link (enl_…) on the console Users page; each user runs it on their own machine:

curl -fsSL https://app.anyray.ai/connect.sh | sh -s -- --enroll https://…/enroll/enl_…

That points their coding tools (Claude Code, Cursor, Codex, …) at the gateway (http://<host>:8787) and mints a personal key bound to their email — no provider keys ever touch a user's machine. For a whole fleet, push one provisioning token via your MDM — see Bulk enrollment with your MDM.

4

Track the migration and confirm the cutover

Watch the Users roster move from Pending to Enrolled; unenrolled /v1 traffic is always rejected. See Configure → Access control.

You're live

Your users' everyday AI tools now run through Anyray — spend attributed by user/team/model, content handled per your privacy mode, the optimizer cutting cost on the hot path.

Troubleshoot

SSH connection issues:

docker compose version                       # Compose v2 present locally (it drives the remote)
DOCKER_HOST=ssh://user@<vm> docker version   # SSH tunnel reaches the remote Docker daemon

View logs on the remote host:

DOCKER_HOST=ssh://user@<vm> docker compose logs <service>

Upgrade

DOCKER_HOST=ssh://user@<vm> docker compose pull
DOCKER_HOST=ssh://user@<vm> docker compose up -d

Soft updates (image-only releases) apply automatically by default — the gateway triggers the bundled updater itself, so the commands above are only needed for hard updates (the update notice in the console's sidebar Updates panel says which, and what to set first) or after opting out. The stack tracks the moving stable channel; pin ANYRAY_IMAGE_TAG=vX.Y.Z in .env to hold a version. See Configure → Updates to turn automatic updates off (console: the sidebar Updates panel).

Configuration

Remote Docker uses the same .env controls as Local / VM: public TLS, rate limits, image mirrors, and a managed Postgres endpoint (ANYRAY_SPEND_DB_URL; ANYRAY_OBSERVABILITY_DB_URL only to split traces onto a separate DB). Set them in the local .env, then re-apply:

DOCKER_HOST=ssh://user@<vm> docker compose up -d

Keep .env backed up and out of Git — it holds the admin key, the content-encryption key required by the default encrypted mode, and the Postgres password.

System requirements

The whole stack's real footprint is roughly 1 vCPU / ~1.5 GB — a small host is plenty. Size up only for high trace/spend volume (it's Postgres that grows).

Tier	vCPU / RAM	AWS	GCP	Azure
Recommended	2 / 4 GB	t3.medium	e2-medium	Standard_B2ms
Minimum / demo	2 / 2 GB	t3.small	e2-small	Standard_B2s