Routing

One default config decides where requests go; a client can override per request, and conditional lanes match on verified identity.

Set the default on Routing (or GET / PUT /admin/routing-config); a client can override per request with the x-anyray-config header. Routing applies to API-key traffic only; passthrough subscription traffic keeps the user's own token and provider. anyray-default, what anyray-connect sends to mean "gateway, you pick", resolves to ANYRAY_DEFAULT_MODEL (default anthropic/claude-sonnet-4-5); override per alias with ANYRAY_MODEL_ALIASES or PUT /admin/model-aliases.

Mode	Behavior
single	One target for every request.
loadbalance	Spread across targets by weight, sticky per conversation.
fallback	Try targets in order; advance on failure.
conditional	Pick a target from request conditions (below).

Transient failures (502 / 503 / 504, network) get 2 attempts; a 429 is retried for non-streaming requests only. A retry block changes it: { "attempts": 0 } opts out, max 5.

Upstream timeouts

Bound the wait on a provider; set via .env + restart.

Env var	Default	Bounds
ANYRAY_UPSTREAM_CONNECT_TIMEOUT_MS	10000	Connecting to the provider.
ANYRAY_UPSTREAM_HEADERS_TIMEOUT_MS	60000	First response byte; raise for slow providers.
ANYRAY_UPSTREAM_BODY_TIMEOUT_MS	600000	Gap between streamed chunks.

Conditional routing

conditional mode picks a named target from an ordered conditions list, else default (no match and no default → 400):

{ "strategy": { "mode": "conditional",
    "conditions": [ { "query": { "metadata.team": "research" }, "then": "frontier" } ],
    "default": "economy" },
  "targets": [ { "name": "frontier", "provider": "anthropic" },
               { "name": "economy", "provider": "openai" } ] }

A query matches metadata.<key>, params.<field>, and url.pathname, with operators ($eq, $ne, $gt / $gte / $lt / $lte, $in, $nin, $regex) combined by $and / $or. First match wins.

Identity fields (metadata.user / userId, metadata.team / teamId, metadata.agent / agentId) are evaluated on the verified key-bound identity, never the header, so a client can't steer an identity lane by editing x-anyray-metadata; every other metadata.<key> field stays client-controlled. The team is the seat's current SCIM-mapped team when SCIM manages the roster, otherwise the team set at enrollment. A bound key with no team ignores a header-supplied team: team lanes need team-bearing keys (SSO enrollment groupTeamMap / defaultTeam).

Pinning a target to a named provider key

Any target may carry provider_key_id naming one of the provider's named keys; the stored config holds the id only, never a credential. With verified team conditions this gives each department its own upstream account:

{ "strategy": { "mode": "conditional",
    "conditions": [ { "query": { "metadata.team": "ml" }, "then": "ml-lane" } ],
    "default": "org" },
  "targets": [ { "name": "ml-lane", "provider": "openai", "provider_key_id": "team-ml" },
               { "name": "org", "provider": "openai" } ] }

Key selection is admin-config only: a provider_key_id inside a client-sent x-anyray-config header is ignored, so a user can't bill another team's key. A save naming an id the provider-key store doesn't hold is rejected; if the key is deleted later, requests routed to that pin fail with 424, never a silent fallback to a different key.

Per-agent routing

An agent (a service key minted on the Agents page) can carry its own provider and named key, so unattended traffic bills a separate upstream account from your developers. Bind it on the agent itself, on the Agents page or at mint time (Add an agent); the Routing page lists such bindings read-only under From Agents. Precedence: the request's own x-anyray-config / x-anyray-provider wins, then the agent's binding, then the org routing config.

When the condition is broader than one agent, express it as a lane instead. metadata.agent is the agent's name and metadata.agentId its key id, which survives a rename:

{ "strategy": { "mode": "conditional",
    "conditions": [ { "query": { "metadata.agent": "release-bot" }, "then": "ci-lane" } ],
    "default": "org" },
  "targets": [ { "name": "ci-lane", "provider": "bedrock", "provider_key_id": "ci-account" },
               { "name": "org", "provider": "openai" } ] }

Both fields derive from the verified key, never the header, so no developer key, shared org key, or BYO pass-through can reach an agent lane; being an agent is a property of the key, not of the name.

Targeted routing on the Routing page

The Routing page builds conditional lanes for you (Targeted routing), wrapping whatever default strategy you already configured. conditional is not one of the strategies in the picker, because it is not a choice you make there: the strategy you pick becomes the default target, and the lanes are the exceptions in front of it. Each lane picks what it matches on:

Match	Condition it writes	Matched against
agent name	metadata.agent	The verified service key's name
agent id (survives rename)	metadata.agentId	The verified service key's id
user	metadata.user	The verified key-bound user
team	metadata.team	The verified key-bound team
model	params.model	The model named in the request

The first four are identity, derived from the key that signed the request; model says what was asked for, never who asked. Match an agent on its id if you may ever rename it: a name lane stops matching on rename and its traffic silently falls back to the default. A lane naming a revoked agent is flagged on the page rather than blocked.

Narrowing a lane with a second test

And also adds a second test to a lane, and both must hold. That is what separates two agents sharing one provider account for some models and holding their own for others.

Say two agents share a Bedrock account, and each has its own xAI key. Two lanes, each matching an agent and the Grok model, pinned to that agent's key. Everything else, Opus included, matches no lane and falls to the base strategy on the shared Bedrock account:

{ "strategy": { "mode": "conditional",
    "conditions": [
      { "query": { "metadata.agentId": "svc_a", "params.model": "grok-4.6" },
        "then": "agentId:svc_a+model:grok-4.6" },
      { "query": { "metadata.agentId": "svc_b", "params.model": "grok-4.6" },
        "then": "agentId:svc_b+model:grok-4.6" }
    ],
    "default": "default" },
  "targets": [
    { "name": "agentId:svc_a+model:grok-4.6", "provider": "x-ai", "provider_key_id": "xai-agent-a" },
    { "name": "agentId:svc_b+model:grok-4.6", "provider": "x-ai", "provider_key_id": "xai-agent-b" },
    { "name": "default", "provider": "bedrock", "provider_key_id": "shared" } ] }

Both tests live in one query object, which the router reads as an AND. The second test offers every dimension except the first: one field holds one value, so two tests on the same dimension could never both match.

Split an agent's models writes that whole set in one step. Name the agent, the models to move, and the account to move them to; it adds one lane per model, matched on the agent's id, above any broader lane already there. Re-running it for the same models re-points them rather than adding a second, unreachable lane.

The page edits only the shape it generates (one or two equality tests per lane). A config with hand-written conditions, a third test, or a $regex is shown read-only rather than flattened on the next save, and stays editable through PUT /admin/routing-config.

A regex lane routes but cannot publish a menu

$regex conditions keep working and are what you want for a whole model family. They just never appear in the model list an agent host reads from GET /v1/me/models, because a regex matches ids without being able to enumerate them. Register the models you want offered as aliases, on the provider's card under Providers.