Deploy on Kubernetes

One Helm chart runs the whole stack: gateway, optimizer, console proxy, Postgres. This page is the install path; chart values, GitOps, external Postgres, sizing, and troubleshooting live in the Kubernetes reference.

Prerequisites

• Kubernetes 1.24+, Helm 3.10+
• A default StorageClass (or set *.storageClass in your values file). A fresh EKS cluster ships none, and some OKE templates strip the default marker: StorageClass setup
• Node capacity for the four pods; a 2-vCPU / 4-GB node fits (system requirements)
• Anyray deployment token (adt_…)

Install

Script-minted secrets: setup.sh quickstart. ArgoCD / Flux: GitOps install.

1

Choose a namespace

export ANYRAY_NAMESPACE="team-ai"   # replace with your target namespace
kubectl create namespace "$ANYRAY_NAMESPACE"

2

Create the anyray-secrets Secret

Author it from the template under The two files you author, or let your own tooling (External Secrets Operator, Sealed Secrets, SOPS) manage it.

kubectl apply -n "$ANYRAY_NAMESPACE" -f anyray-secrets.yaml

3

Write your values file

Non-secret config goes in my-values.yaml:

my-values.yaml

host: "<gateway-ingress-hostname>"               # gateway ingress hostname
gateway:
  publicUrl: https://<gateway-ingress-hostname>
  consolePublicUrl: https://<gateway-ingress-hostname>
  metering:
    enabled: true                    # required: Billing App metering
image:
  tag: policy-stable                 # automatic updates (see Upgrade)
postgres:
  storage: 150Gi                     # 90 days of trace content; IMMUTABLE after install

postgres.storage is required, and immutable afterwards

A fresh install below 150Gi is refused at render time, and the size cannot change later without expanding the PVC by hand; a full volume stops Postgres and the spend and trace stores with it. Floor, opt-out, ArgoCD / Flux caveat: volume size.

4

Install the chart

helm repo add anyray https://charts.anyray.ai
helm install anyray anyray/anyray \
  --version <x.y.z> -f my-values.yaml --namespace "$ANYRAY_NAMESPACE"

List released versions with helm search repo anyray/anyray --versions. The same chart is on Artifact Hub and at oci://public.ecr.aws/anyray/anyray.

5

Expose the gateway and console

Services default to ClusterIP. Expose them to your org network / VPN only, never 0.0.0.0/0. With Ingress the chart routes one host: / and /admin → console proxy, /v1 → gateway:

my-values.yaml (add)

ingress:
  enabled: true
  className: nginx        # your ingress controller
gateway:
  publicUrl: "https://<gateway-ingress-hostname>"
  consolePublicUrl: "https://<gateway-ingress-hostname>"

Apply with another helm upgrade. Keep /admin on the console proxy (it injects the admin key) or console login 401s. The chart pre-tunes the Ingress for streaming; a 60s idle timeout anywhere in front reads as API Error: Connection closed mid-response. TLS, Gateway API, LoadBalancer, NodePort: Exposing services.

6

Verify the install is healthy

kubectl rollout status -n "$ANYRAY_NAMESPACE" deployment/anyray-gateway
kubectl rollout status -n "$ANYRAY_NAMESPACE" deployment/anyray-proxy
curl -fsI https://<gateway-ingress-hostname>/ && echo "console ok"

Open https://<gateway-ingress-hostname>/ and sign in with the admin key (ANYRAY_ADMIN_TOKEN in the anyray-secrets Secret). The gateway base URL for your tools is …/v1.

To finish setup from your coding agent instead of the console, continue with Run Anyray from your AI agent.

The two files you author

An Anyray install is two files you own; setup.sh --k8s only mints the random values.

File	Purpose	Safe to commit?
anyray-secrets.yaml	Kubernetes Secret: admin key, content-encryption key (required by the default encrypted mode), optimizer shared secret, Postgres password, Billing App metering token.	No
my-values.yaml	Helm values: host, storage, image tag, exposure, scheduling, other non-secret settings.	Yes, if it stays secret-free

The chart only reads the key names below from whatever Secret secretName points at, so External Secrets Operator, Sealed Secrets, SOPS, or Vault can own it. Generate ANYRAY_PSEUDONYM_SALT once and keep it stable through upgrades. Values are base64:

anyray-secrets.yaml

apiVersion: v1
kind: Secret
metadata:
  name: anyray-secrets
type: Opaque
data:
  ANYRAY_ADMIN_TOKEN: <base64-admin-token>
  ANYRAY_OPTIMIZER_TOKEN: <base64-optimizer-shared-secret>
  ANYRAY_CONTENT_KEY: <base64-aes-256-gcm-key>
  POSTGRES_PASSWORD: <base64-postgres-password>
  ANYRAY_DEPLOYMENT_TOKEN: <base64-billing-app-deployment-token>
  ANYRAY_PSEUDONYM_SALT: <base64-pseudonym-salt>

Production checklist

Since chart 0.8.0, the gateway, optimizer and console proxy run two replicas by default, with probes, PodDisruptionBudgets and anti-affinity (built-in behavior). The optimizer stays at one on existing installs with optimizer.persistence.enabled: true or an image below v1.10.403 (two-step migration). Plan for at least three 2-vCPU / 4-GB nodes, or a cluster autoscaler. Two changes are yours to make:

1

Run Postgres outside the chart

my-values.yaml

postgres:
  enabled: false
  external:
    databaseUrlSecretKeyRef:
      name: anyray-external-postgres
      key: DATABASE_URL

The bundled Postgres is a one-pod StatefulSet, and the gateway reads client keys from it on the authentication hot path: while it is down, /v1/* fails closed. Use a managed instance with automatic failover (RDS, Cloud SQL, Azure Database) or CloudNativePG in-cluster. Connection strings: managed Postgres; passwordless RDS: IAM auth.

2

Upgrade with --atomic

Add --atomic to every helm upgrade so a bad rollout rolls itself back within progressDeadlineSeconds.

Setting securityContext.runAsUser? Set fsGroup to the same id: the container never starts as root, so it cannot chown the volume itself (details).

Production-shaped

Every service at the default two replicas, Postgres external and highly available, Ingress with the chart's streaming defaults, upgrades with --atomic.

Upgrade

With image.tag: policy-stable (the values file above), images keep themselves current: the tag is a channel every release promotes, the chart forces imagePullPolicy: Always for it, and the autoUpdate CronJob restarts the Deployments nightly. Pods cache the resolved digest, so a moving tag alone never rolls. Schedule, scope, per-component pins: automatic updates.

A roll stalls rather than breaks, and it stalls quietly

maxUnavailable: 0 retires no old pod until its replacement is Ready, so a build that cannot start leaves the old pods serving and the rollout stuck, which kubectl rollout status reports. That includes a hard release needing a new environment variable first: the nightly roll cannot tell it apart and stalls on it at 03:30. The console's Updates panel names exactly what to set.

Roll now:

Roll to the newest build now

kubectl rollout restart deployment -n "$ANYRAY_NAMESPACE" \
  -l app.kubernetes.io/instance=anyray
kubectl rollout status deployment -n "$ANYRAY_NAMESPACE" \
  -l app.kubernetes.io/instance=anyray --timeout=10m

A chart upgrade brings new templates and values, and is what advances pinned images (image.tag unset defaults to the chart's fixed appVersion; a vX.Y.Z pins one build):

helm repo add anyray https://charts.anyray.ai
helm repo update anyray
helm upgrade anyray anyray/anyray \
  --version <chart-version> -f my-values.yaml --namespace "$ANYRAY_NAMESPACE"

Installed from a clone (./helm)? Refresh it with git fetch && git reset --keep origin/main, then helm upgrade anyray ./helm -f my-values.yaml --namespace "$ANYRAY_NAMESPACE".

Verify the running version

kubectl -n "$ANYRAY_NAMESPACE" rollout status deployment/anyray-gateway
kubectl -n "$ANYRAY_NAMESPACE" get deployment anyray-gateway \
  -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'

There is no separate migration step: the gateway self-migrates its schema on boot, serialized across replicas, and durable state (including keys minted by anyray-connect) is shared in Postgres, so upgrades and replica scaling never drop keys (schema and data migrations).