TrueFoundry

The Anyray gateway sits in front of TrueFoundry, which keeps its routing and credentials, as with LiteLLM.

Property	Value
Provider id	truefoundry, never the generic custom upstream (why)
Gateway speaks	Anthropic /v1/messages for Claude, OpenAI /v1/chat/completions otherwise
Credential	A TrueFoundry Personal Access Token or Virtual Account Token
Base URL	Your control plane including /api/llm, or TrueFoundry's SaaS host
Model ids	Namespaced by provider account, such as anthropic-main/claude-opus-5-5

Set it up

1

Allow the TrueFoundry hostname

Add it to ANYRAY_CUSTOM_HOST_ALLOWLIST, or requests fail with "Invalid custom host". Not needed if it is already the configured upstream.

2

Add the provider in the console

Field	Value
Base URL	https://acme.truefoundry.cloud/api/llm, keeping the /api/llm prefix. A trailing /v1 also works.
API key	Your PAT or VAT, sent as a Bearer token
Custom headers	Optional JSON, for TrueFoundry's x-tfy-* routing headers

3

Onboard each developer

Per machine, with the enrollment link and the developer's token:

curl -fsSL https://app.anyray.ai/connect.sh | sh -s -- \
  --enroll https://app.anyray.ai/enroll/enl_… \
  --user dev@example.com \
  --upstream https://acme.truefoundry.cloud/api/llm \
  --upstream-key tfy-synthetic1234 \
  --yes

Mints a personal ark_… key and forwards the token on both lanes. --upstream and --upstream-key go together (CLI commands).

4

Run one session and check the console

Requests appear under truefoundry with the developer's name.

No per-developer tokens? Omit both flags; set one org token on the TrueFoundry provider.

Keep the provider-account prefix on your model ids

Use provider_account/model_name, not a TrueFoundry alias: an unrecognized id gets a flat, wrong price and confuses the cache guard. For short names, use Anyray model aliases.

Request prompt caching with block-level cache_control

Use body cache_control breakpoints (what tools send), not top-level placement or the x-tfy-cache-control header, which the optimizer can't see.

The Responses API needs a routing header

/responses requires x-tfy-provider-name. Set it in Custom headers on the org-key lane; on the per-developer lane, use chat or messages endpoints instead.