Install
Anyray installs into your environment. The whole system — gateway, console, optimizer,
trace backend, and datastores — lives in one monorepo and comes up from a single root
docker-compose.yml. You install and update the stack with docker compose — there is
no installer CLI.
:::info Two different things
Installing the stack is docker compose (this page). Pointing developer tools at the
gateway is a separate, client-side step done with the anyray-connect CLI (or the manual
base-URL exports it automates) — see Developers → Connect.
anyray-connect does not install or run the stack.
:::
Install (docker compose)
# 1. clone the monorepo
git clone https://github.com/anyrayHQ/monorepo.git
cd monorepo
# 2. create your environment file from the template
cp .env.example .env
# then edit .env and set at least:
# ANYRAY_ADMIN_TOKEN — the single admin key for the console + /admin APIs
# ANYRAY_CONTENT_KEY — AES-256 key used to encrypt stored content at rest
# provider keys — e.g. ANYRAY_PROVIDER_KEY_OPENAI / ANYRAY_PROVIDER_KEY_ANTHROPIC
# (or set them later from the console's Providers page)
# 3. bring up the whole stack
docker compose up -d
docker compose up -d brings up:
- the gateway on
:8787— the OpenAI-compatible, multi-provider proxy/router your workloads talk to - the console on
:3000— the admin UI (Spend, Traces, Sessions, Optimizer, Content-privacy) - the optimizer — the request/response optimization hook backend (internal)
- the trace backend — a self-hosted Langfuse-derived trace store (internal)
- datastores — postgres, clickhouse, redis, minio (internal)
Only the gateway (:8787) and the console (:3000) are exposed; everything else is
internal to the compose project.
:::note Running the gateway on the host
The gateway can also run directly on the host instead of in Docker:
cd gateway && npm run build && node build/start-server.js.
:::
Sign in to the console
Open http://localhost:3000 (or http://<host>:3000) and sign in with your
ANYRAY_ADMIN_TOKEN. That single admin key gates the entire console and all /admin/*
APIs — there's no per-page login. From here you configure optimization and content-privacy,
and watch spend, traces, and sessions.
Point your workloads at it
The fastest way to point developer tools (Claude Code, Cursor, Windsurf) and SDK/shell env
at the gateway is the anyray-connect CLI:
npx anyray-connect --gateway http://<host>:8787 --user you@yourorg.com
It writes each tool's base URL and a placeholder key (the real provider key stays
server-side) and is safe to re-run or --revert. See
Developers → Connect.
Or do it manually by overriding each SDK's base URL to the Anyray gateway:
OPENAI_BASE_URL=http://<host>:8787/v1
ANTHROPIC_BASE_URL=http://<host>:8787
Either way you do not edit application code, and there is no org CA, no TLS-MITM, and
no HTTPS_PROXY — it's a plain base-URL redirect. See
Developers → Using SDKs.
:::info Roadmap: zero-touch injection Auto-injecting the endpoint override for you — an admission webhook for in-cluster workers and managed-settings/MDM for laptops — and re-signing Bedrock traffic upstream with the proxy's IRSA role are roadmap. Today you set the base-URL env explicitly. :::
Update
New gateway/optimizer/console logic ships as updated images. To roll it out:
docker compose pull
docker compose up -d
Roll out conservatively: update, send representative traffic, confirm spend and the passthrough rate look right, then widen.
Start in Shadow Mode (roadmap)
The intended adoption path is observe-only Shadow Mode: Anyray sits on the path, changes nothing, and reports what it would save and the measured quality delta on your own traffic — so the "will it hurt our performance?" question gets answered with evidence instead of a promise. See Proof, not promises. (Shadow Mode is roadmap; until it ships, begin with the provably-lossless strategies and a conservative config.)