Deploy on Railway

Deploy the whole stack on Railway from a checked-in config, pinned to a release.

railway config apply provisions all four services from the same published images as every other install and wires the cross-service variables. You generate two public domains, add a provider key, and send traffic; upgrading later is git pull plus another apply.

Prerequisites

• A Railway account with a plan/trial that can run four services (~2–4 GB total; Railway is billed by usage, so there is no host size to pick).
• At least one provider API key (e.g. OpenAI or Anthropic) to add after the stack is up; keys stay server-side on the gateway.
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment).
• The Railway CLI, Node.js, and openssl.

Install

1

Apply the stack

Everything is declared in .railway/railway.ts, whose image tag is bumped in lockstep with every release, so an apply always provisions the current build.

Provision gateway / optimizer / proxy / Postgres

git clone https://github.com/anyrayHQ/install && cd install
npm install            # pulls the Railway IaC SDK (pinned in package.json)

railway login
railway init -n anyray  # or: railway link, to attach an existing empty project
railway config plan     # preview the changes
railway config apply    # create the four services + wire every internal reference

Service	Image	Role
gateway	public.ecr.aws/anyray/gateway:latest	OpenAI-compatible API (:8787)
optimizer	public.ecr.aws/anyray/optimizer:latest	Optimization hook (:8088)
proxy	public.ecr.aws/anyray/proxy:latest	nginx console; rewrites upstreams to *.railway.internal
Postgres	ghcr.io/railwayapp-templates/postgres-ssl:17	Spend + trace store (encrypted at rest)

Traces live in Postgres (anyray_traces / anyray_observations, auto-created via ANYRAY_OBSERVABILITY_DB_URL); the console reads them in-process.

2

Seed secrets and public domains

Railway IaC cannot generate secrets or public domains declaratively; a one-time, idempotent bootstrap seeds both. The secrets (ANYRAY_ADMIN_TOKEN, ANYRAY_CONTENT_KEY, and the rest) are kept as preserve() in railway.ts, so apply never clobbers them.

One-time bootstrap

railway/railway-iac-bootstrap.sh                  # secrets + public domains
railway/railway-iac-bootstrap.sh adt_your_token   # …and connect metering

3

Confirm the public domains

Expose only proxy and gateway; keep optimizer and Postgres private. To check or set one by hand: Settings → Public Networking → Generate domain, targeting:

Service	Target port	This becomes your…
proxy	80 (nginx listens here, not $PORT)	Console URL
gateway	8787	Gateway API URL

Save the two hostnames:

PROXY_DOMAIN=your-proxy.up.railway.app
GATEWAY_DOMAIN=your-gateway.up.railway.app

4

Redeploy the gateway

The config wires the gateway's public URL references:

Service	Variable	Value
gateway	ANYRAY_GATEWAY_PUBLIC_URL	https://${{RAILWAY_PUBLIC_DOMAIN}}
gateway	ANYRAY_CONSOLE_PUBLIC_URL	https://${{proxy.RAILWAY_PUBLIC_DOMAIN}}
gateway	ANYRAY_TRUST_PROXY	true
gateway	ANYRAY_HSTS	true

After generating the two domains, redeploy or restart gateway so Railway resolves those references. If a dashboard publish ever drops them, set the two URL variables manually with the saved hostnames.

Don't swap the two URLs

ANYRAY_GATEWAY_PUBLIC_URL = the gateway domain (port 8787); ANYRAY_CONSOLE_PUBLIC_URL = the proxy domain (port 80). Both live on the gateway service. Swapping them breaks console sign-in redirects. There is no NEXTAUTH_URL in Anyray; if a guide tells you to set it, ignore it.

5

Connect to Anyray Billing app

--connect doesn't apply on Railway, so if the bootstrap didn't set them, put the metering variables on the gateway service directly:

Service	Variable	Value
gateway	ANYRAY_METERING_ENABLED	true
gateway	ANYRAY_DEPLOYMENT_TOKEN	<adt_token>

Once the gateway redeploys and phones home (~10s), the deployment flips to Connected at app.anyray.ai.

6

Add a provider key

Add at least one on the gateway service Variables before sending real traffic, e.g. ANYRAY_PROVIDER_KEY_OPENAI or ANYRAY_PROVIDER_KEY_ANTHROPIC. Multi-field providers (Bedrock, Vertex, Azure) can be configured later from the console Providers page.

7

Open the console and verify

Read ANYRAY_ADMIN_TOKEN from the gateway service's Variables, open https://<proxy-domain>, and sign in with it. Then confirm every leg is healthy:

ADMIN_TOKEN=...   # from the gateway service's Variables

curl -fsS "https://${GATEWAY_DOMAIN}/"                                          # liveness → AI Gateway
curl -fso /dev/null -w "%{http_code}\n" "https://${PROXY_DOMAIN}/anyray-login"  # console → 200
# Deployment health: gateway / observability / spend / optimizer / portal:
curl -fsS "https://${GATEWAY_DOMAIN}/admin/health" -H "Authorization: Bearer ${ADMIN_TOKEN}"

/admin/health returns 503 and names the failing leg if any required service is down. Confirm the Traces view loads in the console; that exercises the observability store end to end.

Upgrade

git pull && railway config apply

The tag in .railway/railway.ts tracks the latest release, so a pull plus an apply rolls the stack forward. Your secrets and generated domains are preserved.

Configuration

Expose exactly two services (proxy on 80, gateway on 8787, the table under Install); never generate public domains for optimizer or Postgres. Optional hardening and traffic controls go on the gateway service's Variables: the same ANYRAY_* rate-limit and body-size vars as every install (Configure → Rate limits).

Let deploys finish their streams

Railway's gap between SIGTERM and SIGKILL defaults to 0 seconds, so a deploy would kill a service instantly and cut every request in flight; on the gateway that means a streaming turn, which a coding assistant reports as API Error: Connection closed mid-response.

RAILWAY_DEPLOYMENT_DRAINING_SECONDS is set per service and must outlive that container's own drain. The config provisions all four (railway/railway.template.json covers one-click deploys); each is a ceiling, not a delay:

Service	Drain	Why this number
gateway	120	Finishes in-flight requests for ANYRAY_SHUTDOWN_DRAIN_MS (default 90s); a streaming completion runs minutes.
optimizer	30	Its own drain is 15s; an optimize call is bounded by the gateway's ceiling on it, not by how long a model takes to answer.
endpoint-control	45	Bounds a single request at 30s and force-exits its drain at 35s.
proxy	60	nginx drains its workers on SIGQUIT while still proxying console responses.

Troubleshoot

Start with /admin/health (see Open the console and verify); it names the failing leg. Then match the symptom below.

Console loads but Traces / dashboard are empty or 502

The gateway proxies fine, but the console can't read observability data. Check the gateway service:

• ANYRAY_OBSERVABILITY_DB_URL (or ANYRAY_SPEND_DB_URL) must reference the Postgres service; the config wires both to ${{Postgres.DATABASE_URL}}.
• Confirm Postgres is running and reachable on the private network.
• curl .../admin/health: an observability leg of {"configured": false} means the DB URL is unset; {"configured": true, "ok": false} means it's set but unreachable.

proxy fails: host not found in upstream "gateway.railway.internal"

The console proxy (nginx) can't resolve the gateway on Railway's private network. The config sets ANYRAY_GATEWAY_HOST on the proxy service and resolves it at request time; confirm it's present (add it if you composed by hand). It also sets ANYRAY_OPTIMIZER_HOST, but the proxy never dials the optimizer; its admin calls go through the gateway. After the gateway service is healthy, redeploy proxy so nginx picks up the now-resolvable host.

Railway private networking is IPv6-only. If an internal service is unreachable, confirm it binds all interfaces (:: or the platform default), not only 0.0.0.0.

A variable shows as a literal ${{...}} reference

Railway reference variables (${{Postgres.DATABASE_URL}}, ${{gateway.ANYRAY_ADMIN_TOKEN}}) resolve only when the service names match exactly (case-sensitive: gateway, optimizer, proxy, Postgres) and the referenced service exists. Fix the name (or re-point the reference) and redeploy the consuming service.

Recover a half-deployed stack

Redeploy in dependency order so each service starts against ready upstreams:

1. Postgres: wait until it's running.
2. gateway, then optimizer (they need Postgres).
3. proxy last (it needs the gateway resolvable).

Railway hands out a fresh internal IPv6 on every redeploy; redeploying proxy after the gateway is the reliable recovery step. Re-run /admin/health to confirm every leg is green.