Deploy on Kubernetes

One Helm chart runs the whole stack: gateway, optimizer, console proxy, Postgres. This page is the install path; chart values, GitOps, external Postgres, sizing, and troubleshooting live in the Kubernetes reference.

Prerequisites

• Kubernetes 1.24+, Helm 3.10+
• A default StorageClass (or set *.storageClass in your values file). A fresh EKS cluster ships none, and some OKE templates strip the default marker: StorageClass setup
• Node capacity for the four pods; a 2-vCPU / 4-GB node fits (system requirements)
• An Anyray deployment token (adt_…) from app.anyray.ai (setup wizard, or Settings → Deployments → New deployment)

Install

Script-minted secrets: setup.sh quickstart. ArgoCD / Flux: GitOps install.

1

Choose a namespace

export ANYRAY_NAMESPACE="team-ai"   # replace with your target namespace
kubectl create namespace "$ANYRAY_NAMESPACE"

2

Create the anyray-secrets Secret

Author it from the template under The two files you author, or let your own tooling (External Secrets Operator, Sealed Secrets, SOPS) manage it.

kubectl apply -n "$ANYRAY_NAMESPACE" -f anyray-secrets.yaml

3

Write your values file

Non-secret config goes in my-values.yaml:

my-values.yaml

host: "<gateway-ingress-hostname>"               # gateway ingress hostname
gateway:
  publicUrl: https://<gateway-ingress-hostname>
  consolePublicUrl: https://<gateway-ingress-hostname>
  metering:
    enabled: true                    # required: Billing app metering
image:
  tag: policy-stable                 # automatic updates (see Upgrade)
postgres:
  storage: 50Gi                      # 90 days of trace content; IMMUTABLE after install

postgres.storage is required, and immutable afterwards

A fresh install below 50Gi is refused at render time, and the size cannot change later without expanding the PVC by hand; a full volume stops Postgres and the spend and trace stores with it. Floor, opt-out, ArgoCD / Flux caveat: volume size.

4

Install the chart

helm install anyray oci://public.ecr.aws/anyray/anyray \
  --version <x.y.z> -f my-values.yaml --namespace "$ANYRAY_NAMESPACE"

List released versions with helm show chart oci://public.ecr.aws/anyray/anyray. Prefer a classic HTTP repo? helm repo add anyray https://charts.anyray.ai serves the same chart, also on Artifact Hub.

5

Expose the gateway and console

Services default to ClusterIP. Expose them to your org network / VPN only, never 0.0.0.0/0. With Ingress the chart routes one host: / and /admin → console proxy, /v1 → gateway:

my-values.yaml (add)

ingress:
  enabled: true
  className: nginx        # your ingress controller
gateway:
  publicUrl: "https://<gateway-ingress-hostname>"
  consolePublicUrl: "https://<gateway-ingress-hostname>"

Apply with another helm upgrade. Keep /admin on the console proxy (it injects the admin key) or console login 401s. The chart pre-tunes the Ingress for streaming; a 60s idle timeout anywhere in front reads as API Error: Connection closed mid-response. TLS, Gateway API, LoadBalancer, NodePort: Exposing services.

6

Verify the install is healthy

kubectl rollout status -n "$ANYRAY_NAMESPACE" deployment/anyray-gateway
kubectl rollout status -n "$ANYRAY_NAMESPACE" deployment/anyray-proxy
curl -fsI https://<gateway-ingress-hostname>/ && echo "console ok"

Open https://<gateway-ingress-hostname>/ and sign in with the admin key (ANYRAY_ADMIN_TOKEN in the anyray-secrets Secret). The gateway base URL for your tools is …/v1.

The two files you author

An Anyray install is two files you own; setup.sh --k8s only mints the random values.

File	Purpose	Safe to commit?
anyray-secrets.yaml	Kubernetes Secret: admin key, content-encryption key (required by the default encrypted mode), optimizer shared secret, Postgres password, Billing app metering token.	No
my-values.yaml	Helm values: host, storage, image tag, exposure, scheduling, other non-secret settings.	Yes, if it stays secret-free

The chart only reads the key names below from whatever Secret secretName points at, so External Secrets Operator, Sealed Secrets, SOPS, or Vault can own it. Generate ANYRAY_PSEUDONYM_SALT once and keep it stable through upgrades. Values are base64:

anyray-secrets.yaml

apiVersion: v1
kind: Secret
metadata:
  name: anyray-secrets
  namespace: team-ai
type: Opaque
data:
  ANYRAY_ADMIN_TOKEN: <base64-admin-token>
  ANYRAY_OPTIMIZER_TOKEN: <base64-optimizer-shared-secret>
  ANYRAY_CONTENT_KEY: <base64-aes-256-gcm-key>
  POSTGRES_PASSWORD: <base64-postgres-password>
  ANYRAY_DEPLOYMENT_TOKEN: <base64-billing-app-deployment-token>
  ANYRAY_PSEUDONYM_SALT: <base64-pseudonym-salt>

Production checklist

The gateway is already redundant: since chart 0.5.0 it defaults to replicas: 2 with no volume (durable state, including provider and user access keys, lives in the shared Postgres), and every workload gets probes, maxUnavailable: 0 rollouts, PodDisruptionBudgets, and anti-affinity (built-in behavior). Three changes are yours to make:

1

Take the optimizer off its volume

my-values.yaml

optimizer:
  persistence:
    enabled: false
  replicas: 2

On its default volume the optimizer upgrades by Recreate: a gap the gateway fails open across, busting the provider prompt cache on every warm session. Optimizer state already lives in the shared Postgres. Requires gateway.contentMode not off (the chart refuses to render) and ANYRAY_CONTENT_KEY set (the chart cannot see it; check yourself). On an existing install, migrate in two steps: two-step migrations.

2

Run Postgres outside the chart

my-values.yaml

postgres:
  enabled: false
  external:
    databaseUrlSecretKeyRef:
      name: anyray-external-postgres
      key: DATABASE_URL

The bundled Postgres is a one-pod StatefulSet, and the gateway reads client keys from it on the authentication hot path: while it is down, /v1/* fails closed. Use a managed instance with automatic failover (RDS, Cloud SQL, Azure Database) or CloudNativePG in-cluster. Connection strings: managed Postgres; passwordless RDS: IAM auth.

3

Upgrade with --atomic

Add --atomic to every helm upgrade so a bad rollout rolls itself back within progressDeadlineSeconds.

Setting securityContext.runAsUser? Set fsGroup to the same id: the container never starts as root, so it cannot chown the volume itself (details).

Production-shaped

Gateway at the default replicas: 2, optimizer off its volume at replicas: 2, Postgres external and highly available, Ingress with the chart's streaming defaults, upgrades with --atomic.

Upgrade

With image.tag: policy-stable (the values file above), images keep themselves current: the tag is a channel every release promotes, the chart forces imagePullPolicy: Always for it, and the autoUpdate CronJob restarts the Deployments nightly. Pods cache the resolved digest, so a moving tag alone never rolls. Schedule, scope, per-component pins: automatic updates.

A roll stalls rather than breaks, and it stalls quietly

maxUnavailable: 0 retires no old pod until its replacement is Ready, so a build that cannot start leaves the old pods serving and the rollout stuck, which kubectl rollout status reports. That includes a hard release needing a new environment variable first: the nightly roll cannot tell it apart and stalls on it at 03:30. The console's Updates panel names exactly what to set.

Roll now:

Roll to the newest build now

kubectl rollout restart deployment -n "$ANYRAY_NAMESPACE" \
  -l app.kubernetes.io/instance=anyray
kubectl rollout status deployment -n "$ANYRAY_NAMESPACE" \
  -l app.kubernetes.io/instance=anyray --timeout=10m

A chart upgrade brings new templates and values, and is what advances pinned images (image.tag unset defaults to the chart's fixed appVersion; a vX.Y.Z pins one build):

helm upgrade anyray oci://public.ecr.aws/anyray/anyray \
  --version <chart-version> -f my-values.yaml --namespace "$ANYRAY_NAMESPACE"

Installed from a clone (./helm)? git pull, then helm upgrade anyray ./helm -f my-values.yaml --namespace "$ANYRAY_NAMESPACE".

Verify the running version

kubectl -n "$ANYRAY_NAMESPACE" rollout status deployment/anyray-gateway
kubectl -n "$ANYRAY_NAMESPACE" get deployment anyray-gateway \
  -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'

There is no separate migration step: the gateway self-migrates its schema on boot, serialized across replicas, and durable state (including keys minted by anyray-connect) is shared in Postgres, so upgrades and replica scaling never drop keys (schema and data migrations).