Deploy locally or on a VM

Run the Anyray stack with docker compose directly on the host.

Prerequisites

Docker Engine (v24+) and Docker Compose v2: docker compose version
~2 GB RAM free
Ports 3000 (console) and 8787 (gateway API) open to your org network or VPN CIDR only, never 0.0.0.0/0
git and openssl on PATH
An Anyray deployment token (adt_…) from app.anyray.ai

Install

Run setup

git clone https://github.com/anyrayHQ/install anyray && cd anyray
./setup.sh --connect <adt_token>
docker compose up -d

setup.sh generates every secret locally into .env, and --connect wires metering to the Billing app. docker compose up -d pulls the published images and starts four services:

Service	Exposed port	What it is
gateway	:8787	OpenAI-compatible multi-provider API
proxy (console)	:3000	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	internal	Request and response optimization hook
postgres	internal	Spend and trace store

Verify

docker compose ps                                       # all services running/healthy
curl -fs http://localhost:8787/ && echo "gateway ok"   # gateway answers

Expected: all four services running, with gateway, optimizer, and postgres (healthy).

Open the console

Open http://<your-host>:3000 and sign in with the admin key setup.sh printed. It is also in .env as ANYRAY_ADMIN_TOKEN.

Traces live in Postgres (anyray_traces and anyray_observations, auto-created) and the console reads them in-process. There is no separate trace datastore to run.

Upgrade

docker compose pull && docker compose up -d

Soft updates apply automatically, because the gateway triggers the bundled updater itself. Run the command above for hard updates, or after opting out (Configure → Updates). The console's Updates panel says which kind a release is. The stack tracks the moving stable channel. Pin ANYRAY_IMAGE_TAG=vX.Y.Z in .env to hold a version.

Configuration

Set these in .env, then run docker compose up -d again.

`.env` variable	What it controls
`ANYRAY_HSTS`	Emits HSTS headers. Set only when serving over HTTPS.
`ANYRAY_ALLOW_PLAINTEXT`	Deploy gate for `plaintext` content capture. The content mode itself is set in the console.
`ANYRAY_RATE_LIMIT_RPM`	Per-identity `/v1` request limit per minute (e.g. `600`).
`ANYRAY_RATE_LIMIT_IP_RPM`	Per-source-IP `/v1` request limit per minute (e.g. `1200`).
`ANYRAY_RATE_LIMIT_UNAUTH_RPM`	Unauthenticated endpoint request limit per minute (e.g. `60`).
`ANYRAY_MAX_CONCURRENT_REQUESTS`	Max simultaneous `/v1` requests per identity or IP (e.g. `20`).
`ANYRAY_MAX_BODY_BYTES`	Max request body size in bytes (e.g. `33554432`).
`ANYRAY_OPTIMIZER_TIMEOUT_MS`	Normal optimizer timeout. Defaults to `800`.

For a public HTTPS gateway, the stack ships a Caddy edge behind the public profile that provisions and renews a Let's Encrypt certificate, with the console and /admin staying in-network: Gateway TLS.

Image mirrors

Use these when your host pulls from an internal registry:

.env
ANYRAY_GATEWAY_IMAGE=registry.example.com/anyray/gateway:latest
ANYRAY_OPTIMIZER_IMAGE=registry.example.com/anyray/optimizer:latest
ANYRAY_PROXY_IMAGE=registry.example.com/anyray/proxy:latest
ANYRAY_POSTGRES_IMAGE=registry.example.com/postgres:17

Managed Postgres

Set ANYRAY_SPEND_DB_URL to use a managed instance instead of the bundled Postgres. The spend store uses it directly and the trace store falls back to it, so one managed database serves both. Set ANYRAY_OBSERVABILITY_DB_URL as well only to split traces onto a separate database.

.env
ANYRAY_SPEND_DB_URL=postgresql://user:password@db.example.com:5432/postgres

The gateway auto-creates the anyray_spend, anyray_traces, and anyray_observations tables on first use. Trace content follows the content mode: AES-256-GCM ciphertext by default, omitted in off, and readable only in deploy-gated plaintext.

Amazon RDS without a stored password

On RDS you can leave the password out of the URL. An RDS endpoint carrying a user and no password uses IAM database authentication: the gateway mints a short-lived token from the machine's own AWS identity on every connection, so .env holds no database credential.

.env
ANYRAY_SPEND_DB_URL=postgresql://appuser@mydb.abc123.us-east-1.rds.amazonaws.com:5432/anyray

Three things must be true on the AWS side. IAM auth is enabled on the instance, the database user was created with GRANT rds_iam TO appuser, and the VM's instance profile grants rds-db:connect on arn:aws:rds-db:<region>:<account>:dbuser:<db-resource-id>/appuser. TLS is configured and verified for you against Amazon's RDS roots, so no sslmode and no CA file. The Kubernetes reference walks the same three steps in more detail.

Containers need an extra IMDS hop. Instance-profile credentials come from the instance metadata service, and EC2 defaults the metadata hop limit to 1, which the Docker bridge network consumes. Raise it to 2 on the instance, or every connection fails to authenticate:

aws ec2 modify-instance-metadata-options \
  --instance-id <instance-id> \
  --http-put-response-hop-limit 2 \
  --http-endpoint enabled

Confirm the resolved mode with GET /admin/health. .spend.auth reads rds-iam when tokens are being minted, and password when something still supplies one. A password in the URL, a ?password= parameter, or PGPASSWORD in the environment all keep password auth, so an existing deployment cannot switch modes by accident.

System requirements

The stack's real footprint is roughly 1 vCPU and ~1.5 GB, so a small host is plenty. Size up only for high trace or spend volume, because it is Postgres that grows.

Tier	vCPU / RAM	AWS	GCP	Azure	OCI
Recommended	2 / 4 GB	`t3.medium`	`e2-medium`	`Standard_B2ms`	`E4.Flex` 1 OCPU / 4 GB
Minimum / demo	2 / 2 GB	`t3.small`	`e2-small`	`Standard_B2s`	`E4.Flex` 1 OCPU / 2 GB

Disk is what grows. The gateway keeps 90 days of trace content by default, and a full disk stops Postgres and both stores with it. Give the host at least 50 GB free behind /var/lib/docker, or the same on a managed instance's storage. Measure your own rate in the first week: Measure datastore growth.

Oracle Cloud shapes: an OCPU is not always a vCPU

On VM.Standard.E4.Flex (AMD) one OCPU is two vCPUs, so a single OCPU already matches the 2-vCPU column. On Ampere VM.Standard.A1.Flex one OCPU is one vCPU, so request 2 to land on the same tier.

Ampere is arm64 and fully supported. Every published image is a multi-arch manifest (linux/amd64 and linux/arm64), so docker compose up -d pulls the matching build with no change to .env. The Always Free allocation (4 OCPU / 24 GB on A1.Flex) covers the recommended tier.

Prerequisites​

Install​

Upgrade​

Configuration​

Amazon RDS without a stored password​

System requirements​