TrueFoundry

Keep TrueFoundry exactly where it is. The Anyray gateway sits in front of it, so every request is optimized and attributed before TrueFoundry routes it upstream.

What's specific to TrueFoundry

Property	Value
Provider id	truefoundry
Gateway speaks	Anthropic /v1/messages for Claude, OpenAI /v1/chat/completions otherwise
Credential	A TrueFoundry Personal Access Token or Virtual Account Token
Base URL	Your control plane including /api/llm, or TrueFoundry's SaaS host
Model ids	Namespaced by provider account, such as anthropic-main/claude-opus-4-8

Requests flow client tool → Anyray gateway → TrueFoundry → provider. The gateway accepts both the Anthropic and OpenAI dialects and forwards each in its own wire format, so client tools stay on their native managed lanes, where the optimizer earns most, while TrueFoundry keeps owning upstream routing and credential management. Everything shared (self-hosting, provider config, credential storage) is in the gateway.

Use the truefoundry provider, not the generic custom upstream

Both reach a TrueFoundry gateway, but only truefoundry forwards Claude requests to its own /v1/messages. The generic anyray-upstream provider translates them into OpenAI chat, which cannot carry a cache_control breakpoint or a thinking block, so a warm Claude session loses the prompt cache TrueFoundry would otherwise pass through to Claude unchanged.

Configure the provider

In the console, add the TrueFoundry provider:

Field	Value
Base URL	https://acme.truefoundry.cloud/api/llm (keep the /api/llm prefix)
API key	Your PAT or VAT, sent as a Bearer token
Custom headers	Optional JSON, for TrueFoundry's x-tfy-* routing headers

A trailing /v1 is fine

TrueFoundry's own setup docs publish the base URL with a /v1 suffix (their ANTHROPIC_BASE_URL example). The gateway appends its own /v1/… path, so it ignores a trailing /v1 on what you paste. Both spellings resolve to the same upstream URL.

warning

The TrueFoundry hostname must be authorized on the gateway, so add it to ANYRAY_CUSTOM_HOST_ALLOWLIST (or TRUSTED_CUSTOM_HOSTS for a private control plane), or every request is rejected with "Invalid custom host".

Onboard a developer

One command per machine. The enrollment link comes from your admin (onboard users); the upstream key is the developer's own TrueFoundry token:

curl -fsSL https://app.anyray.ai/connect.sh | sh -s -- \
  --enroll https://app.anyray.ai/enroll/enl_… \
  --user dev@example.com \
  --upstream https://acme.truefoundry.cloud/api/llm \
  --upstream-key tfy-synthetic1234 \
  --yes

This enrolls the developer (a personal ark_… key, so spend is attributed to them), points every detected tool at the gateway on its managed lane, and writes the passthrough pins that make the gateway forward the developer's token verbatim. --upstream and --upstream-key go together; see CLI commands. Without per-developer tokens, omit both flags and set one org token on the TrueFoundry provider in the console instead.

Success

Run one session from any connected tool, then open the console dashboard: the request appears under the truefoundry provider with the developer's name attached, and the optimizer's savings show within a few turns.

Keep the provider-account prefix on your model ids

TrueFoundry addresses models as provider_account/model_name, and an org can also define its own short aliases. Keep the prefixed form where you can. Two things read the model id, and a house alias is opaque to both: pricing resolves the rate from the id (an unrecognized one falls back to a flat rate and misprices every row), and the optimizer's cache guard classifies the lane from it. Prefixed ids such as anthropic-main/claude-opus-4-8 resolve correctly on both.

If you want short names, define them as model aliases on the Anyray side instead, where the gateway resolves them to a concrete provider and model before either check runs.

Request prompt caching with block-level cache_control

TrueFoundry accepts a cache request three ways: block-level cache_control, an automatic top-level placement, and an x-tfy-cache-control header. Prefer the block-level form, which is what client tools send natively.

The optimizer detects a cached request by scanning for cache_control breakpoints in the body. It cannot see the header form, so a request cached that way reads as uncached, and the optimizer's cache-safety guard will not know there is a warm prefix to protect.

The Responses API needs a routing header

TrueFoundry's /responses endpoint requires an x-tfy-provider-name header naming the provider integration. On the org-key lane you can set it in the provider's Custom headers field. The per-developer passthrough lane has no way to attach it, so Codex-style clients on that lane should use the chat or messages endpoints.