Deploy locally or on a VM

Run the Anyray stack with docker compose directly on the host.

Prerequisites

• Docker Engine (v24+) and Docker Compose v2: docker compose version
• ~2 GB RAM free (the stack's real footprint is ~1.5 GB)
• Ports 3000 (console) and 8787 (gateway API) open to your org network / VPN CIDR only — never 0.0.0.0/0
• git and openssl on PATH
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment) — required for usage metering

Install

1

Install

git clone https://github.com/anyrayHQ/install anyray && cd anyray
./setup.sh --connect <adt_token>
docker compose up -d

setup.sh generates every secret locally into .env; --connect wires the deployment to Anyray Billing app for metering. docker compose up -d pulls the published images and starts:

Service	Exposed port	What it is
gateway	:8787	OpenAI-compatible multi-provider API
proxy (console)	:3000	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	internal	Request/response optimization hook
postgres	internal	Spend + trace store — spend rows are metadata only; trace content per ANYRAY_CONTENT_MODE (encrypted default)

Traces live in Postgres (anyray_traces / anyray_observations, auto-created) and the console reads them in-process — no separate trace datastore to run.

2

Verify the install is healthy

docker compose ps                                       # all services running/healthy
curl -fs http://localhost:8787/ && echo "gateway ok"   # gateway answers

Expected: all four services running; gateway, optimizer, and postgres (healthy).

3

Open the console

Open http://<your-host>:3000 and sign in with the admin key setup.sh printed (also in .env as ANYRAY_ADMIN_TOKEN). Then follow After deployment below.

After deployment — roll out to your users

1

Add a provider key

Provider keys stay server-side — callers never hold them. Add at least one on the console Providers page, or in .env and re-apply:

.env

ANYRAY_PROVIDER_KEY_OPENAI=sk-...        # or ANYRAY_PROVIDER_KEY_ANTHROPIC=sk-ant-...

docker compose up -d

The console path wins over the env var; multi-field providers (Bedrock, Vertex, Azure) are configured on the Providers page. See Configure → Providers.

2

Send a test request

On the Providers page use Send test request, then confirm a row under Spend and a trace under Traces — that exercises routing, attribution, and the trace store end to end.

3

Enroll your users

/v1 is always gated — each user enrolls before their traffic is accepted. Mint an enrollment link (enl_…) on the console Users page; each user runs it on their own machine:

curl -fsSL https://app.anyray.ai/connect.sh | sh -s -- --enroll https://…/enroll/enl_…

That points their coding tools (Claude Code, Cursor, Codex, …) at the gateway and mints a personal key bound to their email — no provider keys ever touch a user's machine. For a whole fleet, push one provisioning token via your MDM — see Bulk enrollment with your MDM.

4

Track the migration and confirm the cutover

Watch the Users roster move from Pending to Enrolled; unenrolled /v1 traffic is always rejected. See Configure → Access control.

You're live

Your users' everyday AI tools now run through Anyray — spend attributed by user/team/model, content handled per your privacy mode, the optimizer cutting cost on the hot path.

Upgrade

docker compose pull && docker compose up -d

Soft updates (image-only releases) apply automatically by default — the gateway triggers the bundled updater itself, so the command above is only needed for hard updates (the update notice in the console's sidebar Updates panel says which, and what to set first) or after opting out. The stack tracks the moving stable channel; pin ANYRAY_IMAGE_TAG=vX.Y.Z in .env to hold a version. See Configure → Updates to turn automatic updates off (console: the sidebar Updates panel).

Configuration

Set these in .env, then run docker compose up -d again.

Public TLS and gateway hardening

For a public HTTPS gateway, put Caddy in front and keep the console/admin surface in-network:

ANYRAY_PUBLIC_DOMAIN=anyray.example.com
ANYRAY_GATEWAY_BIND=127.0.0.1
ANYRAY_TRUST_PROXY=true
ANYRAY_HSTS=true
ANYRAY_GATEWAY_PUBLIC_URL=https://anyray.example.com
ANYRAY_CONSOLE_PUBLIC_URL=http://<your-host>:3000

docker compose --profile public up -d

Optional traffic controls:

ANYRAY_RATE_LIMIT_RPM=600
ANYRAY_RATE_LIMIT_IP_RPM=1200
ANYRAY_RATE_LIMIT_UNAUTH_RPM=60
ANYRAY_MAX_CONCURRENT_REQUESTS=20
ANYRAY_MAX_BODY_BYTES=33554432

Image mirrors

Use these when your host pulls from an internal registry:

ANYRAY_GATEWAY_IMAGE=registry.example.com/anyray/gateway:latest
ANYRAY_OPTIMIZER_IMAGE=registry.example.com/anyray/optimizer:latest
ANYRAY_PROXY_IMAGE=registry.example.com/anyray/proxy:latest
ANYRAY_POSTGRES_IMAGE=registry.example.com/postgres:17

Managed Postgres

To use a managed instance instead of the bundled Postgres, set ANYRAY_SPEND_DB_URL — the spend store uses it directly and the trace store falls back to it, so one managed DB serves both. Set ANYRAY_OBSERVABILITY_DB_URL as well only to split traces onto a separate DB:

.env

ANYRAY_SPEND_DB_URL=postgresql://user:password@db.example.com:5432/postgres

The gateway auto-creates the anyray_spend / anyray_traces / anyray_observations tables on first use. Spend rows are always metadata only; trace content follows ANYRAY_CONTENT_MODE — AES-256-GCM ciphertext by default, omitted in off, readable only in deploy-gated plaintext.

System requirements

The whole stack's real footprint is roughly 1 vCPU / ~1.5 GB — a small host is plenty. Size up only for high trace/spend volume (it's Postgres that grows).

Tier	vCPU / RAM	AWS	GCP	Azure
Recommended	2 / 4 GB	t3.medium	e2-medium	Standard_B2ms
Minimum / demo	2 / 2 GB	t3.small	e2-small	Standard_B2s