Anthropic
Point Claude Code and the Anthropic SDKs at the gateway. Anyray works with your team's own Claude subscriptions, a shared API key, or both at once.
Pick either mode, or run both: each request picks the right one, and both get usage tracking and token savings.
| Property | Value |
|---|---|
| Provider id | anthropic |
| Gateway speaks | Messages API → api.anthropic.com/v1/messages |
| Credential | Shared API key, or the developer's own login |
| Client base URL | ANTHROPIC_BASE_URL → http://<gateway>:8787 |
API key
Point your client at the gateway; Anyray uses a shared key.
export ANTHROPIC_BASE_URL=http://<gateway>:8787
Add the key in Console → Providers → anthropic (or set ANYRAY_PROVIDER_KEY_ANTHROPIC).
This mode supports routing, fallbacks, and load-balancing across your Claude traffic.
The shared key never reaches Anthropic's admin API: /v1/organizations/* gets a 400.
Routed Anthropic targets keep the client's anthropic-beta header (or x-anyray-anthropic-beta).
Precedence: the target's anthropic_beta, then the body's anthropic_beta, then the client header.
Betas reserved for subscription credentials are removed on this lane.
Prompt cache lifetime and keep-warm
Claude Code asks for an hour-long prompt cache on its main thread. An hour-long write costs 2x the input price against 1.25x for 5 minutes. By default the gateway sends the client's cache settings as received, on this lane and on subscription login. No setting here changes what the model reads.
The cache lifetime and Keep the cache fresh are held off: markers go to the provider exactly as
the client sent them, nothing refreshes them, the card reads Held off, and the prompt-cache
fields of PUT /admin/v1/settings answer 400. The settings below describe the lane once it returns.
Set it in Console → Optimizer → Prompt cache, or as promptCacheTtl on
PUT /admin/v1/settings:
| Setting | What it does |
|---|---|
client (default) | The client's cache settings go exactly as sent. This is the off switch. |
5m | Every request of the session is cached for 5 minutes. A pause longer than that writes the whole prompt again, so use it with Keep the cache fresh on. |
1h | New sessions are cached for an hour on this lane, and on Bedrock for Claude 4.5 and later. The card shows Recommended when your last 7 days of sessions would have returned at least 1.5x the extra cost, with 3 sessions in 4 no worse off. |
auto | Starts each new session at 5m, or at 1h once your last 7 days show pauses of 5 to 60 minutes common enough that the hour would save at least 1.5x its extra write cost. A session with no keep-warm (this lane, or a seat not opted in) is priced as writing again after every pause over 5 minutes. Stays at 5m for models whose cache reads cost under a tenth of input, and until 20 sessions are measured. |
A session takes its lifetime on its first request. A live session can move from 1 hour to 5 minutes
at no cost, but never back: an hour-long request writes again whatever the session cached at 5
minutes. So 1h reaches new sessions only, and 5m applies at once. Subagents keep their own
5-minute cache. A client can send x-anyray-cache-ttl: 5m or 1h for one request, under the same
rule. Under client the header is ignored.
For main-thread requests lowered to 5m, the gateway keeps that tier through optimization and
retries, and the payback check prices eligible writes at it.
A rolling deploy onto this behavior, a rollback, and switching to client can each send a session
hour-long markers after 5-minute ones. Each writes again, at most once, the entries written under
the other lifetime. Plan them for a quiet hour.
Upgrading moves a deployment from 5m or auto to client once and turns Keep the cache
fresh off, including the subscription-seat opt-in: the client's markers go as sent and nothing
pings, as before September 30. A session running at 5 minutes writes its prompt again once. An
admin can choose 5m, auto or refreshes again afterwards; a replica that cannot read the
setting runs client.
Keep-warm. On a pass-through request (the client's own credential), while a tool runs, the gateway can re-send the session's last request as a cache read with no output just before the cache expires, so the next step reads the prompt instead of writing it again. Refreshes run only while your sessions' pauses make them pay, and stop once one shows the cache was lost.
- A subscription seat is refreshed only when
promptCacheKeepWarmSubscriptionis on (off by default). The refreshes count toward its plan limits. A rate-limited seat is never refreshed. - A pass-through API key is billed per token, so it follows
promptCacheKeepWarm. - With keep-warm off, the gateway still allows for a refresh a replica may have started before the switch: for three hours it attests a paused session cold only past that reach. After that the session's own cache lifetime decides, so the optimizer can price a resumed session's first turn at the cold rate.
- This lane, with the shared key, has no keep-warm. Only the cache lifetime applies.
Keep the cache fresh defaults to Off on every lane. Upgrading resets existing deployments to
Off once; an admin can turn it on in the prompt-cache settings. promptCacheKeepWarm is off
(default), tool, or tool+idle, which also refreshes a session waiting on its user for up to
promptCacheKeepWarmIdleMaxMinutes (30 by default).
Every refresh is billed on its own spend row and never counts as a seat or toward a user's token
cap. With several gateway replicas, only the one that served the session's newest request sends
them. x-anyray-cache-keep-warm can shorten or stop them for one request, never extend them.
GET /admin/v1/settings/prompt-cache returns the current settings, the recommendation, the pause
data keep-warm uses, and what auto picks.
Subscription login
A developer signed into a Claude plan (Claude Code on Claude / Max) keeps their own login. Plain Connect sets it up:
npx anyray-connect@latest
Add --subscription to force it for every tool. Either way, each developer needs their own
gateway key (ark_…, from --enroll or --sso; see client keys).
Anyray sends the request to Anthropic with your own token and Claude Code's original shape, no shared key involved. Requests are still tracked and optimized, but this mode skips routing and fallbacks (it rides your subscription token).
Why the request keeps Claude Code's exact shape
Anthropic throttles a subscription token if the request doesn't look like real Claude Code.
So pass-through (marked x-anyray-auth-mode: passthrough) keeps the client's own headers
(user-agent, anthropic-*, x-anthropic-*, x-app*, x-claude-code-*,
x-claude-remote-*, x-cc-*, x-client-*, x-stainless-*) and strips edge headers (ALB,
Railway, Cloudflare, service mesh). Anthropic sees Claude Code, not Anyray. The token is
never logged or stored, and Anyray still compresses the request body for token savings. A
custom x-anyray-custom-host keeps its own headers for corporate proxies.
How rate limits (429s) are handled
On a 429, the gateway pauses that seat's Claude requests. Short waits are handled
server-side; longer ones come back as 429 with Retry-After. Tune the max wait with
ANYRAY_ANTHROPIC_THROTTLE_WAIT_MS (60000 ms default; 0 to never wait).
Seat overflow (optional)
Off by default. When a seat runs out of subscription capacity, the gateway can retry on a
server-held API key so work keeps flowing. Those retries bill as subscription-extra and are
capped per hour.
Set it through the admin API as a subscription fallback: the Anthropic keys that take over when a seat is throttled, for everyone, a team, or one person.
A seat with no fallback at any level keeps its 429. The old seatOverflow setting is
retired. If it was on, the gateway turned it into the same org-wide fallback on upgrade (your
Anthropic key, same hourly cap), unless one was already set.
Overflow only ever goes to api.anthropic.com; requests sent to a custom host
(x-anyray-custom-host) never use the org key.
One rate limit is never absorbed. When Anthropic refuses an over-200k request from a seat that has no long-context usage credits, Claude Code reads that refusal to drop its own context window back to 200k. The gateway surfaces it verbatim rather than overflowing, so the client corrects itself instead of continuing to send a context the seat cannot afford.