Private-registry / restricted-egress install

Pull every image from your own registry; outbound HTTPS to Billing only.

Not a disconnected deployment

Anyray does not support a fully disconnected runtime. Every deployment connects to app.anyray.ai with an adt_… token for metering. Allow only this required Billing App egress, plus the vendor hosts for any spend connectors you enable.

For clusters that must pull every image from an internal, admission-gated registry: one global.imageRegistry value repoints every image, and a BOM script lists exactly what to mirror.

Prerequisites

• An internet-connected "jump" host with helm 3.10+ and crane (or skopeo), used once to pull and mirror artifacts across the gap.
• Your internal container registry (Harbor / ECR / Artifactory / Nexus) reachable from the cluster, with push credentials.
• The target Kubernetes cluster (1.24+), a default StorageClass, and an image pull secret for your registry if it requires auth.
• Outbound HTTPS from the gateway to the Anyray Billing App, directly or through a forward proxy.
• An Anyray deployment token (adt_…) from app.anyray.ai, used at runtime for mandatory metering.

Optional spend connectors need their vendor hosts: api.cursor.com, api.devin.ai, or api.github.com plus copilot-reports.github.com. Allow only the connectors you enable.

Install

1

Generate the image bill-of-materials (BOM)

List the exact images the chart deploys, digest-pinned for reproducible mirrors:

git clone https://github.com/anyrayHQ/install anyray && cd anyray
./scripts/gen-image-bom.sh > anyray-images.txt

anyray-images.txt

docker.io/postgres@sha256:0af6…                # docker.io/postgres:17
public.ecr.aws/anyray/gateway@sha256:14ee…     # …/gateway:v1.10.94
public.ecr.aws/anyray/optimizer@sha256:f092…   # …/optimizer:v1.10.94
public.ecr.aws/anyray/proxy@sha256:e190…       # …/proxy:v1.10.94

No install-repo clone in your pipeline? The list is just helm template anyray anyray --repo https://charts.anyray.ai --version <x.y.z> | grep 'image:'.

2

Mirror the images into your registry

Copy each image into your internal registry, preserving the path so the chart's override lines up:

REG=my.registry:5000
grep -v '^#' anyray-images.txt | while read -r img _; do
  crane cp "$img" "$REG/${img#*/}"   # public.ecr.aws/anyray/gateway@… → my.registry:5000/anyray/gateway@…
done

(skopeo copy docker://<src> docker://<dst> works the same. If the registry network cannot reach public registries, use crane pull/docker save on the jump host, carry the image tarball in, then crane push/docker load.)

3

Pull the chart to a local archive

helm pull anyray --repo https://charts.anyray.ai --version <x.y.z>   # → anyray-<x.y.z>.tgz

Pin <x.y.z> to a released chart version (helm search repo anyray/anyray --versions after helm repo add anyray https://charts.anyray.ai). Carry anyray-<x.y.z>.tgz across the gap alongside the mirrored images.

4

Install from your registry

One value repoints every image at your registry:

helm install anyray ./anyray-<x.y.z>.tgz -n anyray --create-namespace \
  --set global.imageRegistry=my.registry:5000 \
  -f my-values.yaml

If your registry needs credentials, create a pull secret and reference it in your values:

my-values.yaml

global:
  imageRegistry: my.registry:5000    # or pass via --set as above
image:
  pullSecrets:
    - name: my-registry-creds

Author my-values.yaml and the anyray-secrets Secret exactly as in the Kubernetes guide; neither needs registry access.

Success

kubectl get pods -n anyray shows every pod pulling from my.registry:5000. An ImagePullBackOff means the chart version and your mirrored tags are out of step; the chart's appVersion must have a matching image in your registry.

Behind a forward proxy

The optimizer, updater, database, and private addresses always bypass the proxy.

1

Allow the gateway's hosts on the proxy

Host	Used for
app.anyray.ai	Metering (required) and SSO sign-in
Your AuthKit domain (<org>.authkit.app, or the custom auth domain set in WorkOS)	Organization MCP sign-in
Each LLM provider host you use, such as api.anthropic.com, api.openai.com, or bedrock-runtime.<region>.amazonaws.com	Inference
sts.<region>.amazonaws.com; for an SSO profile also portal.sso.<region>.amazonaws.com and oidc.<region>.amazonaws.com	Bedrock credentials from an assumed role, EKS web identity, or SSO profile
The MCP servers and alert webhooks you configure	Connector calls and threshold notifications
The spend connector hosts you enable	Vendor spend polling
github.com (optional)	The Connect update check. Without it, Connect updates are not offered

2

Refresh the compose file (Docker Compose)

On an existing install, run the Upgrade command first: older compose files do not pass these variables through.

3

Point the gateway at the proxy

NO_PROXY lists only your own internal hosts, such as a private model endpoint. Add HTTP_PROXY only for a plain http:// upstream. If STS goes through a VPC endpoint, or your proxy does not allow it, add sts.<region>.amazonaws.com to NO_PROXY; otherwise each Bedrock credential lookup waits out the proxy.

.env (Docker Compose)

.env (Docker Compose)

HTTPS_PROXY=http://proxy.example.com:3128
NO_PROXY=.corp.example.com

my-values.yaml (Helm)

my-values.yaml (Helm)

gateway:
  extraEnv:
    - name: HTTPS_PROXY
      value: http://proxy.example.com:3128
    - name: NO_PROXY
      value: .corp.example.com

my-values.yaml (Helm, proxy with a password)

my-values.yaml (Helm, proxy with a password)

# Keep the password out of Git: store the URL in a Secret first.
#   kubectl create secret generic anyray-proxy -n anyray \
#     --from-literal=url=http://user:password@proxy.example.com:3128
gateway:
  extraEnv:
    - name: HTTPS_PROXY
      valueFrom:
        secretKeyRef:
          name: anyray-proxy
          key: url
    - name: NO_PROXY
      value: .corp.example.com

Apply with docker compose up -d or helm upgrade.

4

Confirm the gateway uses it

Check egress

curl --fail-with-body -sS "https://<your-gateway>/admin/health" \
  -H "Authorization: Bearer $ANYRAY_ADMIN_TOKEN" \
  | jq '{egressProxy, metering: .portal.leaseAcquire.lastOutcome}'

Output

{
  "egressProxy": {
    "configured": true,
    "https": "proxy.example.com:3128",
    "noProxyEntries": 1
  },
  "metering": "lease"
}

Success

egressProxy shows host:port only, never credentials. metering: lease means the proxy works.

Make the proxy refuse internal destinations

The gateway refuses names that resolve locally to private, loopback, or metadata addresses. The proxy resolves each name again, so a name whose DNS answer changes can still reach one. Deny loopback, RFC 1918, link-local, CGNAT (100.64.0.0/10), and metadata destinations on the proxy. In Squid, before any http_access allow:

squid.conf

acl to_private dst 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 169.254.0.0/16 127.0.0.0/8 ::1 fc00::/7 fe80::/10
http_access deny to_private

A TLS-inspecting proxy sees keys and prompts

Inspection decrypts provider API keys and every prompt and response. Exclude the provider hosts and app.anyray.ai from inspection, and the AWS STS, SSO portal, and SSO OIDC hosts above: they carry temporary credentials and tokens. Hosts it still inspects need its root CA on the gateway.

Mounting that CA, and the full proxy rules: Restricted-egress reference.

Secrets, GitOps, and upgrades

• Secrets stay yours. The anyray-secrets Secret is managed by your own tooling (External Secrets Operator, Sealed Secrets, SOPS), and no vendor script runs in your cluster. Keys are listed under The two files you author.
• GitOps. Host anyray-<x.y.z>.tgz in your own chart registry and point an ArgoCD Application / Flux HelmRelease at it with global.imageRegistry in the values. See the ArgoCD example.
• Upgrades. Re-run the BOM for the new version, mirror the new images, helm pull the new chart, then helm upgrade: the same four steps.