TrueFoundry

The gateway sits in front of your TrueFoundry AI Gateway, which keeps owning upstream routing and credentials.

Same front-door arrangement as LiteLLM: requests flow client tool → Anyray gateway → TrueFoundry → provider, each dialect forwarded in its own wire format so client tools stay on their native managed lanes. What differs here:

Property	Value
Provider id	truefoundry
Gateway speaks	Anthropic /v1/messages for Claude, OpenAI /v1/chat/completions otherwise
Credential	A TrueFoundry Personal Access Token or Virtual Account Token
Base URL	Your control plane including /api/llm, or TrueFoundry's SaaS host
Model ids	Namespaced by provider account, such as anthropic-main/claude-opus-4-8

Use the truefoundry provider, not the generic custom upstream

Only truefoundry forwards Claude requests to TrueFoundry's own /v1/messages; the generic anyray-upstream provider translates them into OpenAI chat and loses the prompt cache a warm Claude session depends on.

Configure the provider

In the console, add the TrueFoundry provider:

Field	Value
Base URL	https://acme.truefoundry.cloud/api/llm (keep the /api/llm prefix)
API key	Your PAT or VAT, sent as a Bearer token
Custom headers	Optional JSON, for TrueFoundry's x-tfy-* routing headers

A trailing /v1 is fine

TrueFoundry's own setup docs publish the base URL with a /v1 suffix (their ANTHROPIC_BASE_URL example). The gateway appends its own /v1/… path, so it ignores a trailing /v1 on what you paste. Both spellings resolve to the same upstream URL.

warning

The TrueFoundry hostname must be authorized on the gateway, so add it to ANYRAY_CUSTOM_HOST_ALLOWLIST (a control plane you configure as the gateway's upstream needs no allowlist entry, private address included), or every request is rejected with "Invalid custom host".

Onboard a developer

One command per machine. The enrollment link comes from your admin (onboard users); the upstream key is the developer's own TrueFoundry token:

curl -fsSL https://app.anyray.ai/connect.sh | sh -s -- \
  --enroll https://app.anyray.ai/enroll/enl_… \
  --user dev@example.com \
  --upstream https://acme.truefoundry.cloud/api/llm \
  --upstream-key tfy-synthetic1234 \
  --yes

This enrolls the developer (a personal ark_… key, so spend is attributed to them), points every detected tool at the gateway on its managed lane, and writes the passthrough pins that make the gateway forward the developer's token verbatim. --upstream and --upstream-key go together; see CLI commands. Without per-developer tokens, omit both flags and set one org token on the TrueFoundry provider in the console instead.

Success

Run one session from any connected tool, then open the console dashboard: the request appears under the truefoundry provider with the developer's name attached, and the optimizer's savings show within a few turns.

Keep the provider-account prefix on your model ids

TrueFoundry addresses models as provider_account/model_name, and an org can also define its own short aliases. Keep the prefixed form where you can. Two things read the model id, and a house alias is opaque to both: pricing resolves the rate from the id (an unrecognized one falls back to a flat rate and misprices every row), and the optimizer's cache guard classifies the lane from it. Prefixed ids such as anthropic-main/claude-opus-4-8 resolve correctly on both.

If you want short names, define them as model aliases on the Anyray side instead, where the gateway resolves them to a concrete provider and model before either check runs.

Request prompt caching with block-level cache_control

TrueFoundry accepts a cache request three ways: block-level cache_control, an automatic top-level placement, and an x-tfy-cache-control header. Prefer the block-level form, which is what client tools send natively.

The optimizer detects a cached request by scanning for cache_control breakpoints in the body. It cannot see the header form, so a request cached that way reads as uncached, and the optimizer's cache-safety guard will not know there is a warm prefix to protect.

The Responses API needs a routing header

TrueFoundry's /responses endpoint requires an x-tfy-provider-name header naming the provider integration. On the org-key lane you can set it in the provider's Custom headers field. The per-developer passthrough lane has no way to attach it, so Codex-style clients on that lane should use the chat or messages endpoints.