Deploy locally or on a VM

Run the Anyray stack with docker compose directly on the host.

Prerequisites

Docker Engine (v24+) and Docker Compose v2: docker compose version
~2 GB RAM free
Ports 3000 (console) and 8787 (gateway API) open to your org network or VPN CIDR only, never 0.0.0.0/0
git and openssl on PATH
Anyray deployment token (adt_…)

Install

Run setup

git clone https://github.com/anyrayHQ/install anyray && cd anyray
./setup.sh --connect <adt_token>
docker compose up -d

setup.sh generates every secret locally into .env, and --connect wires metering to the Billing App. docker compose up -d pulls the published images and starts four services:

Service	Exposed port	What it is
gateway	:8787	OpenAI-compatible multi-provider API
proxy (console)	:3000	Admin console (Spend, Traces, Optimizer, Privacy)
optimizer	internal	Request and response optimization hook
postgres	internal	Spend and trace store

Verify

docker compose ps                                       # all services running/healthy
curl -fs http://localhost:8787/ && echo "gateway ok"   # gateway answers

All four services read running, with gateway, optimizer, and postgres (healthy).

Open the console

Open http://<your-host>:3000 and sign in with the admin key setup.sh printed. It is also in .env as ANYRAY_ADMIN_TOKEN.

To finish setup from your coding agent instead of the console, continue with Run Anyray from your AI agent.

Traces live in Postgres (anyray_traces and anyray_observations, auto-created) and the console reads them in-process. There is no separate trace datastore to run.

Upgrade

git fetch && git reset --keep origin/main && docker compose pull && docker compose up -d --remove-orphans

Run it in the install checkout. The git step refreshes the compose files and keeps your .env, which is untracked, so --remove-orphans can drop a service a release retires.

The gateway triggers the bundled updater itself, so soft updates apply on their own. Run the command above for hard updates, or after opting out (Configure → Updates). The console's Updates panel says which kind a release is. The stack tracks the moving stable channel. Pin ANYRAY_IMAGE_TAG=vX.Y.Z in .env to hold a version.

Configuration

Set these in .env, then run docker compose up -d again.

`.env` variable	What it controls
`ANYRAY_HSTS`	Emits HSTS headers. Set only when serving over HTTPS.
`ANYRAY_ALLOW_PLAINTEXT`	Deploy gate for `plaintext` content capture. The content mode itself is set in the console.
`ANYRAY_RATE_LIMIT_RPM`	Per-identity `/v1` request limit per minute (e.g. `600`).
`ANYRAY_RATE_LIMIT_IP_RPM`	Per-source-IP `/v1` request limit per minute (e.g. `1200`).
`ANYRAY_RATE_LIMIT_UNAUTH_RPM`	Unauthenticated endpoint request limit per minute (e.g. `60`).
`ANYRAY_MAX_CONCURRENT_REQUESTS`	Max simultaneous `/v1` requests per identity or IP (e.g. `20`).
`ANYRAY_MAX_BODY_BYTES`	Max request body size in bytes (e.g. `33554432`).
`HTTPS_PROXY`	Corporate forward proxy for the gateway's outbound calls (e.g. `http://proxy.example.com:3128`). See Behind a forward proxy.
`HTTP_PROXY`	The same, for plain `http://` upstreams. Leave unset unless you route to one.
`NO_PROXY`	Your own internal hosts the gateway reaches without the proxy (e.g. `.corp.example.com`).

For a public HTTPS gateway, the stack ships a Caddy edge behind the public profile that provisions and renews a Let's Encrypt certificate, with the console and /admin staying in-network: Gateway TLS.

Managed Postgres

Set ANYRAY_SPEND_DB_URL to use a managed instance instead of the bundled Postgres. The spend store uses it directly and the trace store falls back to it, so one managed database serves both. Set ANYRAY_OBSERVABILITY_DB_URL as well only to split traces onto a separate database.

.env
ANYRAY_SPEND_DB_URL=postgresql://user:password@db.example.com:5432/postgres

The gateway auto-creates the anyray_spend, anyray_traces, and anyray_observations tables on first use. Trace content follows the content mode: AES-256-GCM ciphertext by default, omitted in off, and readable only in deploy-gated plaintext.

Amazon RDS without a stored password

An RDS endpoint carrying a user and no password uses IAM database authentication. The gateway mints a short-lived token from the machine's own AWS identity on every connection, so .env holds no database credential.

.env
ANYRAY_SPEND_DB_URL=postgresql://appuser@mydb.abc123.us-east-1.rds.amazonaws.com:5432/anyray

Three things must be true on the AWS side. IAM auth is enabled on the instance, the database user was created with GRANT rds_iam TO appuser, and the VM's instance profile grants rds-db:connect on arn:aws:rds-db:<region>:<account>:dbuser:<db-resource-id>/appuser. TLS is configured and verified for you against Amazon's RDS roots, so no sslmode and no CA file. The Kubernetes reference walks the same three steps in more detail.

Containers need an extra IMDS hop. Instance-profile credentials come from the instance metadata service, and EC2 defaults the metadata hop limit to 1, which the Docker bridge network consumes. Raise it to 2, or every connection fails to authenticate:

aws ec2 modify-instance-metadata-options \
  --instance-id <instance-id> \
  --http-put-response-hop-limit 2 \
  --http-endpoint enabled

Confirm the resolved mode with GET /admin/health. .spend.auth reads rds-iam when tokens are being minted, and password when something still supplies one. A password in the URL, a ?password= parameter, or PGPASSWORD in the environment all keep password auth, so an existing deployment cannot switch modes by accident.

System requirements

The real footprint is about 1 vCPU and 1.5 GB, so a small host is plenty. Size up only for high trace or spend volume, because it is Postgres that grows.

Tier	vCPU / RAM	AWS	GCP	Azure	OCI
Recommended	2 / 4 GB	`t3.medium`	`e2-medium`	`Standard_B2ms`	`E4.Flex` 1 OCPU / 4 GB
Minimum / demo	2 / 2 GB	`t3.small`	`e2-small`	`Standard_B2s`	`E4.Flex` 1 OCPU / 2 GB

Disk is what grows. The gateway keeps 90 days of trace content by default, and a full disk stops Postgres and both stores with it. Give the host at least 200 GB free behind /var/lib/docker. With managed Postgres, give the database at least 150 GB and the host at least 50 GB. Measure your own rate in the first week: Measure datastore growth.

Oracle Cloud shapes: an OCPU is not always a vCPU

On VM.Standard.E4.Flex (AMD) one OCPU is two vCPUs, so a single OCPU already matches the 2-vCPU column. On Ampere VM.Standard.A1.Flex one OCPU is one vCPU, so request 2 to land on the same tier.

Ampere is arm64 and fully supported. Every published image is a multi-arch manifest (linux/amd64 and linux/arm64), so docker compose up -d pulls the matching build with no change to .env. The Always Free allocation (4 OCPU / 24 GB on A1.Flex) covers the recommended tier.

Prerequisites​

Install​

Upgrade​

Configuration​

Amazon RDS without a stored password​

System requirements​