Deploy on Railway

Launch the whole stack on Railway with one click.

The template provisions all four services from the same published images as every other install and wires the cross-service variables. You generate two public domains, add a provider key, and you're live.

Prerequisites

• A Railway account with a plan/trial that can run four services (~2–4 GB total).
• At least one provider API key (e.g. OpenAI or Anthropic) to add after the stack is up — keys stay server-side on the gateway.
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment) — required for usage metering.

Install

1

Deploy on Railway

Click to open Railway's one-click template and accept the new-project prompt. Railway creates all four services and auto-generates every secret (ANYRAY_ADMIN_TOKEN, 64-hex ANYRAY_CONTENT_KEY, the Postgres password). First deploy takes ~3 min.

Deploy on Railway

Prefer the CLI, or a repeatable install?

Infrastructure-as-Code below deploys the same stack from a checked-in .railway/railway.ts that tracks each release — the version-pinned, upgrade-friendly path. Or Compose the template yourself (Railway CLI) builds the four services from the template spec.

2

What you're deploying

Railway is managed and billed by usage; there is no host size to pick.

Service	Image	Role
gateway	public.ecr.aws/anyray/gateway:latest	OpenAI-compatible API (:8787)
optimizer	public.ecr.aws/anyray/optimizer:latest	Optimization hook (:8088)
proxy	public.ecr.aws/anyray/proxy:latest	nginx console; rewrites upstreams to *.railway.internal
Postgres	ghcr.io/railwayapp-templates/postgres-ssl:17	Spend + trace store (metadata only, encrypted at rest)

Traces live in Postgres (anyray_traces / anyray_observations, auto-created via ANYRAY_OBSERVABILITY_DB_URL) and the console reads them in-process — no separate trace datastore.

3

Generate public domains

Expose only proxy and gateway — keep optimizer and Postgres private. For each service: Settings → Public Networking → Generate domain, targeting:

Service	Target port	This becomes your…
proxy	80 (nginx listens here, not $PORT)	Console URL
gateway	8787	Gateway API URL

Save the two hostnames:

PROXY_DOMAIN=your-proxy.up.railway.app
GATEWAY_DOMAIN=your-gateway.up.railway.app

4

Redeploy the gateway

The template wires the gateway's public URL references:

Service	Variable	Value
gateway	ANYRAY_GATEWAY_PUBLIC_URL	https://${{RAILWAY_PUBLIC_DOMAIN}}
gateway	ANYRAY_CONSOLE_PUBLIC_URL	https://${{proxy.RAILWAY_PUBLIC_DOMAIN}}
gateway	ANYRAY_TRUST_PROXY	true
gateway	ANYRAY_HSTS	true

After generating the two domains, redeploy or restart gateway so Railway resolves those references. If a dashboard publish ever drops them, set the two URL variables manually with the saved hostnames.

Don't swap the two URLs

ANYRAY_GATEWAY_PUBLIC_URL = the gateway domain (port 8787); ANYRAY_CONSOLE_PUBLIC_URL = the proxy domain (port 80). Both live on the gateway service. Swapping them breaks console sign-in redirects. There is no NEXTAUTH_URL in Anyray — if a guide tells you to set it, ignore it.

5

Connect to Anyray Billing app

--connect doesn't apply on Railway, so set the metering variables on the gateway service directly (adt_… token from app.anyray.ai — the setup wizard mints it, or use Settings → Deployments → New deployment):

Service	Variable	Value
gateway	ANYRAY_METERING_ENABLED	true
gateway	ANYRAY_DEPLOYMENT_TOKEN	<adt_token>

Once the gateway redeploys and phones home (~10s), the deployment flips to Connected at app.anyray.ai.

6

Add a provider key

Add at least one on the gateway service Variables before sending real traffic — e.g. ANYRAY_PROVIDER_KEY_OPENAI or ANYRAY_PROVIDER_KEY_ANTHROPIC. Multi-field providers (Bedrock, Vertex, Azure) can be configured later from the console Providers page.

7

Open the console and verify

Read ANYRAY_ADMIN_TOKEN from the gateway service's Variables, open https://<proxy-domain>, and sign in with it. Then confirm every leg is healthy:

ADMIN_TOKEN=...   # from the gateway service's Variables

curl -fsS "https://${GATEWAY_DOMAIN}/"                                          # liveness → AI Gateway
curl -fso /dev/null -w "%{http_code}\n" "https://${PROXY_DOMAIN}/anyray-login"  # console → 200
# Deployment health — gateway / observability / spend / optimizer / portal:
curl -fsS "https://${GATEWAY_DOMAIN}/admin/health" -H "Authorization: Bearer ${ADMIN_TOKEN}"

/admin/health returns 503 and names the failing leg if any required service is down; a healthy stack returns "ok": true with every leg green. Confirm the Traces view loads in the console — that exercises the observability store end to end.

Private network is IPv6-only

Railway private networking is IPv6-only. The proxy reaches the gateway over gateway.railway.internal (resolved at request time via ANYRAY_GATEWAY_HOST, set by the template); optimizer admin calls ride the gateway's capability-gated proxy. If an internal service is unreachable, confirm it binds all interfaces (:: or the platform default), not only 0.0.0.0. See Troubleshoot below.

After deployment — roll out to your users

What goes in Railway vs. what you run locally

In Railway lives the deployment — services, provider keys, variables. On each user's machine you run anyray-connect to point their coding tools at the gateway's public URL; nothing about the tools is configured in Railway.

1

Confirm a provider key

You added one during install. Add more on the gateway service Variables (ANYRAY_PROVIDER_KEY_<SLUG>) or the console Providers page (multi-field providers go there). See Configure → Providers.

2

Send a test request

On the console Providers page use Send test request, then confirm a row under Spend and a trace under Traces — that exercises routing, attribution, and the trace store end to end.

3

Enroll your users

/v1 is always gated — each user enrolls before their traffic is accepted. Mint an enrollment link (enl_…) on the console Users page; each user runs it on their own machine:

curl -fsSL https://app.anyray.ai/connect.sh | sh -s -- --enroll https://…/enroll/enl_…

That points their coding tools (Claude Code, Cursor, Codex, …) at the gateway (https://<gateway-domain>) and mints a personal key bound to their email — no provider keys ever touch a user's machine. For a whole fleet, push one provisioning token via your MDM — see Bulk enrollment with your MDM.

4

Track the migration and confirm the cutover

Watch the Users roster move from Pending to Enrolled; unenrolled /v1 traffic is always rejected (401). See Configure → Access control.

You're live

Your users' everyday AI tools now run through Anyray — spend attributed by user/team/model, content handled per your privacy mode, the optimizer cutting cost on the hot path.

Infrastructure-as-Code

Deploy the whole stack from a checked-in .railway/railway.ts instead of the dashboard template — reproducible, and pinned to a release tag that tracks each upgrade.

Reach for this when you want a version-controlled, reproducible install (or manage several Anyray projects). The image tag in .railway/railway.ts is bumped in lockstep with every release, so railway config apply always provisions the current build — nothing to keep current by hand.

1

Prerequisites

The Railway CLI, Node.js, and openssl.

2

Apply the stack

Provision gateway / optimizer / proxy / Postgres

git clone https://github.com/anyrayHQ/install && cd install
npm install            # pulls the Railway IaC SDK (pinned in package.json)

railway login
railway init -n anyray  # or: railway link — attach an existing empty project
railway config plan     # preview the changes
railway config apply    # create the four services + wire every internal reference

3

Seed secrets and public domains

Railway IaC can't generate secrets or public domains declaratively, so a one-time, idempotent bootstrap seeds them — the generated secrets (ANYRAY_ADMIN_TOKEN, ANYRAY_CONTENT_KEY, …, kept as preserve() in railway.ts so apply never clobbers them) plus the gateway (:8787) and console (:80) domains and their URL variables:

One-time bootstrap

railway/railway-iac-bootstrap.sh                  # secrets + public domains
railway/railway-iac-bootstrap.sh adt_your_token   # …and connect metering

Then add a provider key and open the console exactly as in Add a provider key and Open the console and verify above.

4

Upgrade later

git pull && railway config apply

The tag in .railway/railway.ts tracks the latest release, so a pull + apply rolls the stack forward. Your secrets and generated domains are preserved.

Which path?

The one-click template is fastest for a first try — every secret auto-generated, no CLI — but it's best-effort on version: the marketplace template is updated by hand, so it can trail the latest release by a version or two. Infrastructure-as-Code is the recommended path for real / production installs and staying current: the pinned tag lives in your repo, CI bumps it every release, and railway config apply rolls the stack forward — always the latest build.

Compose the template yourself (Railway CLI)

Railway templates are composed in the dashboard — use this path to build the stack from the template spec yourself. It stands up the same four services; once deployed, rejoin the steps above at Generate public domains.

1

Install and authenticate the Railway CLI

bash <(curl -fsSL railway.com/install.sh) -y
railway login
railway whoami

2

Create the project

railway init -n anyray
railway open

In the dashboard, create the four services from the template spec: images, variables, start commands, health checks, and volume mount paths. Keep the service names exactly as written (gateway, optimizer, proxy, Postgres) — Railway reference variables are name-based and case-sensitive. Secrets are auto-generated via Railway template functions; the gateway's ANYRAY_OBSERVABILITY_DB_URL references the Postgres service (${{Postgres.*}}).

Confirm these persistent volume mounts before deploy — the gateway and optimizer write all runtime state under ANYRAY_DATA_DIR (/data in the template):

Service	Mount path
gateway	/data
optimizer	/data
Postgres	/var/lib/postgresql/data

3

Deploy and confirm status

railway service status --all

gateway, optimizer, and proxy should be healthy or successful; Postgres running. Then follow Generate public domains onward above — the CLI equivalents:

railway domain --service proxy --port 80 --json
railway domain --service gateway --port 8787 --json

railway redeploy --service gateway --yes

# Connect to Anyray Billing app (required for metering) — adt token from app.anyray.ai
railway variable set --service gateway \
  ANYRAY_METERING_ENABLED=true \
  ANYRAY_DEPLOYMENT_TOKEN="<adt_token>"

printf '%s' "$OPENAI_API_KEY" | \
  railway variable set --service gateway --stdin ANYRAY_PROVIDER_KEY_OPENAI

railway variable list --service gateway --kv | grep '^ANYRAY_ADMIN_TOKEN='

Configuration

Public exposure

Expose exactly two services — never generate public domains for optimizer or Postgres:

Service	Public port	Purpose
proxy	80	Admin console and gated /admin/* APIs
gateway	8787	OpenAI-compatible /v1/* API

Optional hardening and traffic controls (on the gateway service):

ANYRAY_RATE_LIMIT_RPM=600
ANYRAY_RATE_LIMIT_IP_RPM=1200
ANYRAY_RATE_LIMIT_UNAUTH_RPM=60
ANYRAY_MAX_CONCURRENT_REQUESTS=20
ANYRAY_MAX_BODY_BYTES=33554432

Troubleshoot

Start with /admin/health (see Open the console and verify) — it names the failing leg. Then match the symptom below.

Console loads but Traces / dashboard are empty or 502

The gateway proxies fine, but the console can't read observability data. Check the gateway service:

• ANYRAY_OBSERVABILITY_DB_URL (or ANYRAY_SPEND_DB_URL) must reference the Postgres service — the template wires both to ${{Postgres.DATABASE_URL}}.
• Confirm Postgres is running and reachable on the private network.
• curl .../admin/health — an observability leg of {"configured": false} means the DB URL is unset; {"configured": true, "ok": false} means it's set but unreachable.

proxy fails: host not found in upstream "gateway.railway.internal"

The console proxy (nginx) can't resolve the gateway on Railway's private network. The template sets ANYRAY_GATEWAY_HOST on the proxy service and resolves it at request time — confirm it's present (add it if you composed by hand). The template also sets ANYRAY_OPTIMIZER_HOST, but the proxy never dials the optimizer — its admin calls go through the gateway. After the gateway service is healthy, redeploy proxy so nginx picks up the now-resolvable host.

A variable shows as a literal ${{...}} reference

Railway reference variables (${{Postgres.DATABASE_URL}}, ${{gateway.ANYRAY_ADMIN_TOKEN}}) resolve only when the service names match exactly (case-sensitive: gateway, optimizer, proxy, Postgres) and the referenced service exists. Fix the name (or re-point the reference) and redeploy the consuming service.

Recover a half-deployed stack

Redeploy in dependency order so each service starts against ready upstreams:

1. Postgres — wait until it's running.
2. gateway, then optimizer (they need Postgres).
3. proxy last (it needs the gateway resolvable).

Railway hands out a fresh internal IPv6 on every redeploy; redeploying proxy after the gateway is the reliable recovery step. Re-run /admin/health to confirm every leg is green.