Routing

People and agents can have their own routes. Requests without one follow the org config and its conditional lanes.

A new node-written origin, or a new pairing of credentials with a base URL or custom_endpoints, needs providerkeys:manage (providers:write for API keys) as well as routing:write. This covers roots, groups and every branch, including nodes without a provider. A host injected from the node's own provider key needs no extra grant. Copying that URL into routing, or inheriting it for another node's credential, is node-written.

The comparison uses dispatch's key injection and inheritance. It distinguishes named pins, inline bags and unpinned credentials; headers add a separate keyed MAC. These MACs stay inside the save comparison. Provider keys come from authority; unavailable authority returns 503 provider_keys_authority_unavailable. Operators can retain existing grants. A new grant without permission returns 403 outbound_host_requires_providerkeys_manage and audits only the actor, code and route.

Where	How
Org default	Routing page, or GET / PUT /admin/routing-config
One person	Users → person → Routing (below)
Per request	x-anyray-config header

API-key traffic only; passthrough traffic (a subscription seat, or a caller's own provider key) keeps its own token and provider. It still runs the config's guardrails (pass-through traffic).

Routing reads mask inline provider credentials and customBody, including snake-case spellings, as {"$anyrayCredentialRef":"<64 lowercase hex HMAC>"}. inlineCredentials: {present, targets} reports presence and node count. Stored config and dispatch stay unchanged.

Guardrail checks are masked the same way: every field under a check's credentials except addresses and modes (resourceName, customHost, azureAuthMode, base URLs, domains, regions), plus a top-level apiKey, a webhook or log sink's headers, and every URL a check sends data to (webhookURL, logURL, jwksUri): an incoming-webhook URL is itself a credential. An address that embeds userinfo, a query or a fragment is masked whole. A reference stays on the check it came from, in the same guardrail field and spelling. Changing that check's plugin, any of its parameters or their key order is refused, and so is a reference placed anywhere the read did not put one. You can remove, reorder or disable (is_enabled) checks. New literal guardrail credentials are still accepted.

Echo references unchanged. They resolve from the authoritative config and bind the node's raw fields, spellings and types, except weight and request_timeout. Ancestors bind dispatch's propagated fields; group-local strategy and on_status_codes remain editable. You can reorder or remove targets and edit independent siblings. No routing save, even a security admin's, can change a reference's addressing. A target with its own credential and no provider_key_id still takes the fields it leaves unset, Base URL included, from the provider's default key, which only providerkeys:manage can change. New literal customBody values remain writable; new credentials must use Provider keys and provider_key_id. A save replies with the config as stored, read like a GET, so the reply saves back unchanged. If that read fails, the save still happened and the reply is 503 routing_config_saved_unreadable; reload before saving again.

Saves check schema and depth, refuse literal credentials, compare expectedRevision, resolve and bind references, check reload parity, then authorize outbound tuples using resolved credentials. Resolution and authorization share one stored snapshot, fenced by the write's revision. Referenced nodes and ancestors keep raw own fields; other subtrees normalize host spellings. An absent or empty root api_key may also have a reference so previous releases reject projected saves safely. When guardrail credentials are masked, the first unused root credential spelling holds one too.

Save refusal	Code
New literal credentials	400 inline_credentials_refused
A sealed value: a string starting rsec:v<n>:, or an anyray_sealed object	400 routing_sealed_value_refused
Unknown or stale reference	400 routing_credential_reference_invalid
Changed bound fields	400 routing_credential_address_changed
Changed guardrail check under a reference	400 guardrail_credential_check_changed
Conflicting aliases	400 routing_field_alias_conflict
Config would change or disappear on reload	400 routing_config_load_mismatch
Saved, but reading it back failed (reload)	503 routing_config_saved_unreadable
No database, and the routing file can't be read or parsed (GET too)	500 routing_config_unreadable

The private routing key is a durable random root in the shared database, with a data-dir file only when no database is configured. Rotating the admin token does not change revisions or references. If the key is unavailable, reads and saves return 503 routing_credential_key_unavailable without a revision; requests keep using cached routing. A read the store could not serve returns the cached copy with fromCache: true. Restore the shared spend database or writable ANYRAY_DATA_DIR, then reload.

Stored routing credentials can be sealed with a key derived from ANYRAY_ADMIN_TOKEN. One this token cannot open fails only its own target with 424, never another key, and /admin/health reports routingCredentials: unreadable. Replace that target's credential with a provider key.

Setting	Value
anyray-default (sent by anyray-connect)	Resolves to ANYRAY_DEFAULT_MODEL, default anthropic/claude-sonnet-5-5. With an org default config saved, when every target the request can reach (in that config, or in the person's or agent's own routing) serves that model, routing picks the provider and key as it would for the model id itself; Bedrock counts for a Claude model. Otherwise, and on a deployment with no default config, it goes to the model's provider. An alias you set for anyray-default always uses its own provider.
Aliases	ANYRAY_MODEL_ALIASES or PUT /admin/model-aliases
Retries	502 / 503 / 504 and network errors: 2 attempts. 429: non-streaming only. retry block: { "attempts": 0 } opts out, max 5

Mode	Behavior
single	One target for every request.
loadbalance	Spread across targets by weight, sticky per conversation.
fallback	Try targets in order; advance on failure.
conditional	Pick a target from request conditions (below).

Upstream timeouts

Set in .env, then restart.

Env var	Default	Bounds
ANYRAY_UPSTREAM_CONNECT_TIMEOUT_MS	10000	Connecting to the provider.
ANYRAY_UPSTREAM_HEADERS_TIMEOUT_MS	60000	First response byte of a streaming request.
ANYRAY_UPSTREAM_BODY_TIMEOUT_MS	600000	Gap between streamed chunks, and the wait for a non-streaming response's headers.

Conditional routing

First matching conditions entry wins, else default; neither gives a 400.

{ "strategy": { "mode": "conditional",
    "conditions": [ { "query": { "metadata.team": "research" }, "then": "frontier" } ],
    "default": "economy" },
  "targets": [ { "name": "frontier", "provider": "anthropic" },
               { "name": "economy", "provider": "openai" } ] }

Query part	Values
Fields	metadata.<key>, params.<field>, url.pathname
Operators	$eq, $ne, $gt / $gte / $lt / $lte, $in, $nin, $regex; combine with $and / $or

Regex limits

$regex is admin config only: a client x-anyray-config carrying one gets a 400. Saving refuses a pattern over 128 characters, a backreference or lookbehind, a repeated group that itself repeats or alternates, and a pattern that could take too long to match. A pattern that starts with ^ can hold more repeats (^\d+-\d+$ saves, .*.*x does not). A $regex never matches a value over 256 characters, and a stored pattern that breaks these rules never matches.

A regexMatch or regexReplace guardrail rule has the same limits, and a client x-anyray-config may carry one. The flags are g, i and s, each at most once:

Check	Rule	Flags
regexMatch	the pattern as written (/api/v1/ matches the slashes)	a flags parameter: { "rule": "secret", "flags": "i" }
regexReplace	/pattern/flags, or a bare pattern; redactText up to 1,024 characters	in the rule: /secret/gi (a flags parameter gets a 400)

Each rule is judged on a 100,000-character prompt, so one that slows on long text (a.*b, grok.*fast) gets a 400 on the request and is refused on save, while \d{3}-\d{2}-\d{4}, \d+ and sk-[A-Za-z0-9]{48} pass. A config may carry at most 32 regexMatch and regexReplace checks. The x-anyray-default-input-guardrails and x-anyray-default-output-guardrails headers count toward the 32 of the x-anyray-config sent with them, and get 32 of their own without one.

One request's regex checks share a fixed amount of matching work, across every fallback or load-balance target that reruns them and every config they come from. A request that would go past it gets a 446 and is not sent on, so text never goes through unchecked or unredacted. The same 446 stops a regexReplace whose redactText would grow the text by more than its own length plus 64 KiB.

A jsonSchema guardrail's pattern and patternProperties keys have the same limits, read with the u flag the validator uses, and share the same matching work. A schema can't use the url format (it can take too long on long text): use uri. A schema that could apply too many subschemas to one JSON value (allOf of two $refs to the next level, repeated), or is too large to load, gets a 400 and is refused on save. A recursive schema, such as a tree node whose children refer back to it, is fine. One request's jsonSchema checks also share a fixed amount of validation work.

What checks read

Request	Input checks read	Output checks read
Chat completions, messages	The latest turn	The answer's text
Responses (/v1/responses)	The latest turn	The answer's output_text and refusals
Completions	prompt	The answer's text
Embeddings	input	Nothing to check
Token counts	The latest turn	Nothing to check
Image generations and edits	prompt	Refused, as below

The latest turn is every message after the last assistant message, plus an assistant prefill that ends the request. That covers text blocks, tool results, text documents, search results, OpenAI tool messages, and a system message that follows the user's. On the Responses API it is what the client added after the model's last response (its reasoning, message and tool calls): tool outputs (function_call_output, shell stdout and stderr, and the rest) and new messages, plus the prompt template's variables. The system prompt, instructions and earlier turns are not read, except by a redaction (see Redaction). Partner guardrails get the same text, tool output included.

Any other body is one the checks cannot read: a file upload, audio, a batch, a request through the /v1/* proxy, or an image, a file or any answer that is not JSON. Under a guardrail with deny, or a check that can rewrite it, it gets a 446 and is not sent on. A GET or DELETE sends no body, so input checks have nothing to read on it. Checks that read only the request type, the metadata or a header (allowedRequestTypes, requiredMetadataKeys, jwt) run on every request, and a hook made only of them never refuses a body for being unreadable.

Input checks judge the request as the client sent it. Optimizing an allowed request never changes their result: a wordCount minimum the client's text meets still passes when the optimized text is shorter, and a maximum it exceeds still refuses the request, however short the optimized text would be.

On chat completions, completions, messages and the Responses API, the first target's input checks run before the optimizer. The optimizer sees only what they let through, and a refused request is never optimized. The first attempt reuses their result. If a check rewrote a value the target's override_params sets, that attempt runs the checks again. A fallback target runs its own input checks when the request reaches it.

Which guardrails run

The guardrails that run are those of the config that routes the request. A request's x-anyray-config, its x-anyray-provider, or the signing person's or agent's own routing replaces the org config, guardrails included: the org config's guardrails do not run on that request. On a passthrough request, x-anyray-provider only names the provider the caller's key belongs to, so it replaces nothing (pass-through traffic).

In a config, a target's hooks and guardrails run in addition to the ones it inherits, never instead of them, so a fallback, load-balance or conditional target cannot skip a rule set above it.

When a check fails

A check that throws fails its hook. With deny, or a check that can rewrite, the request gets a 446; otherwise the check is recorded as failed and the request goes on.

These checks can rewrite what they read: regexReplace, addPrefix, webhook, anyray.pii, azure.pii, bedrock.guard, pangea.pii, promptfoo.pii, patronus.pii, patronus.phi, f5-guardrails.scan, acuvity.scan, crowdstrike-aidr.guardChatCompletions and exa.online. Every other check only passes or fails.

A check that finds no text to read (an answer that only calls tools, an empty prompt) passes with no error, whatever failOnError says. An answer whose only text is a refusal, or whose text sits past its first choice, does not count as having no text. The checks do not read that text, so such an answer gets the same result it always did.

A check that returns an error (a partner it cannot reach, missing credentials) fails if the check sets failOnError: true. A guardrail with deny then blocks the request with a 446, and the check's error in hook_results reads the check returned an error. Without failOnError, the error is recorded and the request goes on.

Set failOnError on a deny guardrail you rely on

With failOnError, a partner outage blocks requests until the partner is back. Without it, a request can slip past: one that makes the partner call time out causes the error itself.

Under failOnError, a check that can rewrite blocks with a 446 when it returns an error, even in a mutator or a guardrail without deny. An async hook runs too late to block. Any other check in a hook without deny records the failure, and the request or answer goes on.

failOnError takes true or false. Any other value (the string "false", 1) gets a 400, in an x-anyray-config, in the default-guardrail headers and when an admin saves the routing config. A config saved before this check that holds such a value still loads, and the value counts as false.

A config saved before these limits still loads. Each rule runs on as much text as it can finish in time: about 100,000 characters for a rule saved under these limits, with the turn's text blocks counted together. Past that, when a text block is not a string (a token-id prompt, say), or when a jsonSchema check can't validate its JSON in time (nested too deeply, an array too long for uniqueItems, more JSON than the request's validation work allows, or a saved schema these limits now refuse), a guardrail with deny blocks the request with a 446. A regexReplace always does, and so does one that fails to run, so unchecked or unredacted text never goes through. A guardrail without deny records the check as failed.

The hook_results the gateway adds to a response list each hook's id, type, verdict, deny, transformed and skipped, and each check's id, verdict and an error with a fixed reason. They never include the text or JSON a check matched, or its rule.

Redaction

A regexReplace redaction covers more than the latest turn: the system prompt and every message (on the Responses API, instructions and every input item), rewritten the same way every turn. A client sends its own unredacted copy of the history back each turn, so this keeps a secret out of later requests, and it keeps the bytes the provider caches identical turn to turn. Signed thinking blocks and tool-call inputs are left as they are; an OpenAI refusal is redacted like any text. A redacted answer loses the log probabilities of the text that changed. A redaction writes its result back to the places it read, an empty result included, and a failOnDetection one fails only on a match in the latest turn.

Redactions add up, across hooks and whether or not a hook is sequential. A regexReplace that read text another redaction has since changed runs again on the new text. A partner redaction is never called twice: when it lands on changed text, the regexReplace redactions run again on its result, and the request gets a 446 if one of them can't. An embeddings request is checked and redacted too, its inputs counted as one text (a card-number rule reaches about 106,000 characters across the batch).

For a request from a verified user, each gateway replica remembers where a rule matched each text (never the text), for that user only and until it goes unused for an hour, so a session pays for its history's regex work once. A long session that reaches a replica that has not seen it is checked in full there and can pass the rule's limit: that request gets a 446, but the texts that fit are remembered, so sending it again gets further. A request with no verified user is checked in full every time.

A partner redaction (anyray.pii, azure.pii, pangea.pii, promptfoo.pii, patronus.pii, patronus.phi, bedrock.guard, f5-guardrails.scan and acuvity.scan, with redaction on) covers the same places for a request from a verified user. The latest turn goes to the partner, and only it decides the check's verdict. Each replica remembers, for that user and until it goes unused for an hour, where the partner changed each text and what it put there, never the text itself. A hash or format-preserving replacement is kept as the partner returned it, which is what the provider gets anyway. History the replica has seen is rewritten the same way with no partner call. History it has not seen goes to the partner in separate calls, oldest first: for each check, at most 512 texts, 2,000,000 characters and 10 seconds a request, with 8 calls at a time.

Past those limits, or when the partner fails on the history, the check returns an error. With failOnError, the request gets a 446 and the next one picks up where it stopped, so a long session on a replica that has not seen it is refused until enough of its history is remembered. A 1,500-text session takes 3 requests when each partner call takes 120 ms, and 21 at 1.2 seconds for a partner that checks one text a call; after that, the session is not refused again on that replica while it stays in use. Without failOnError, history the partner did not check goes on as the client sent it. A request with no verified user has only its latest turn redacted: its history reaches the provider as the client sent it, and each redacted turn costs one prompt-cache miss when it becomes history. A crowdstrike-aidr check already sends the whole request and replaces it with the partner's answer.

Azure's PII service takes at most 5 texts of 5,120 characters a call, counting an emoji as one. A longer text is cut at its last newline before the limit, else its last space, else its last punctuation mark, else at the limit, and each piece is checked; an entity that spans a cut, such as a two-word name, can be missed. A text counts as checked only when every piece comes back. A latest turn of more than 512 pieces is not checked.

Azure checks

An Azure check (azure.contentSafety, azure.pii, azure.shieldPrompt, azure.protectedMaterial) in a request's x-anyray-config or default-guardrails header runs on the caller's own API key or Entra client credentials. One that sets azureAuthMode to managed or workload gets a 400, because those modes use the gateway's own Azure identity. In the org routing config, managed works (workload identity is not supported by these checks). The gateway mints that identity's token for Azure AI services only and sends it only to an Azure AI services host (*.openai.azure.com, *.cognitiveservices.azure.com or *.services.ai.azure.com). A save whose check names another customHost, or a resourceName that moves the host off Azure, gets a 400, and the check refuses it with an error before fetching the token. The gateway verifies the TLS certificate of every Azure host, custom or not.

A bedrock.guard check reads awsAccessKeyId, awsSecretAccessKey, awsRegion and an optional awsSessionToken from its credentials, or awsAuthType: assumedRole with awsRoleArn and awsExternalId. A check saved with the older names accessKeyId, accessKeySecret and region works too. When a check has both, the aws name wins.

A caller's bedrock.guard check with awsAuthType: assumedRole gets a 400: it would assume the role with the gateway's own AWS identity. Send the caller's own AWS access key pair.

Streamed answers

On chat completions, completions, messages and the Responses API, a streamed response is held until it ends and checked whole when an output guardrail has deny or an output check can rewrite the answer. A denial is the same 446 JSON a non-streamed one returns, and a redacted answer is sent as a stream carrying the redacted text. A held stream the gateway cannot read, or cannot rebuild after a redaction, gets a 446, and so does a Responses stream with an event whose text the checks do not read. A held stream past 64 MB gets a 446 too, and the gateway stops reading it. When a held stream ends before the checks get an answer to read, the 446 error type says why: upstream_stream_failed when the upstream broke it, held_stream_cut when the gateway stopped reading (the size cap, or a client that left). Other streams are sent as they arrive. A single stream event longer than 128 MB (as many characters, on a text stream), held or not, ends the stream before that event, the way a dropped upstream connection ends it, and the gateway stops reading it. The spend row of a refused answer carries what the provider billed for it, and the refusal adds no savings. When a fallback or retry then moves on, the row adds that cost, at the refused target's price, to the answer's.

Codex gets a guardrail denial and a held_stream_cut as a 400 with the same body, on an API key and on a subscription seat. Codex retries a 446 up to five times, and each retry is a new answer the provider bills and the gateway refuses again; a 400 it shows at once. An upstream_stream_failed stays a 446, so Codex retries a broken upstream. To tell a denial apart, the gateway reads at most 64 KB of a 446 body, for at most two seconds; a longer or slower body reaches Codex as sent.

Identity fields

Identity fields read the verified key, not the x-anyray-metadata header:

Field	Reads
metadata.user, metadata.userId	The key's bound user
metadata.team, metadata.teamId	The key's bound team
metadata.agent, metadata.agentId	The service key's name, and its id

Identity lanes need per-person keys

On a shared org key or BYO pass-through credential, metadata.user comes from the header, so anyone can claim a user lane.

Other metadata.<key> fields are client-controlled. Team = SCIM-mapped team, else the enrollment team; a key with no team ignores a header team (SSO enrollment groupTeamMap / defaultTeam).

Guardrails on pass-through traffic

A passthrough request keeps its own token and provider and is never routed through the config: a Claude Code, Codex or Copilot seat, or any request sent with x-anyray-auth-mode: passthrough and the caller's own key. It runs the config's guardrails and nothing else of it (no key, retry, cache or override): the hooks the org default config, or the signing agent's own routing, gives that request. A caller's x-anyray-config replaces the org's, guardrails included.

Check	On pass-through
Input deny	446; the request never reaches the upstream
Input redaction	The request is rewritten before it leaves, retries included
Output deny or redaction	A successful answer is held until it ends and checked whole, as on API-key traffic, then sent, sent redacted, or refused with a 446. An upstream error is sent as it came
A body the checks cannot read	446; on Copilot, also a request to a path that is not chat completions, Messages or Responses while an output deny or redaction applies

As on an API key, Codex gets a guardrail denial or a held_stream_cut as a 400.

When the config's router picks no target for a seat request (no condition matches and there is no default, or every load-balance weight is 0), the seat runs the guardrails set on that router and above it, and none of its targets'. With none there, the request goes as if no guardrail were set. A route that fails for another reason (a condition or the default names a target that doesn't exist, say) still gets a 446, as it fails on API-key traffic.

Every check runs, so a webhook, log or partner check in the config gets the request's text, as it does for API-key traffic. With no guardrails, pass-through traffic is forwarded with no added work.

On Claude Code and Codex, a caller's x-anyray-config that breaks a config rule gets the same 400 as on any other route. Copilot reads a caller's config only for its guardrails, so only one that carries guardrails gets that 400; one without them is ignored.

Checks that only record (a log hook, a guardrail without deny) run on the answer as they do for API-key traffic: a streamed answer is not held for them, and a failed one turns a JSON answer into a 246. They don't run on Codex's WebSocket.

Codex's WebSocket cannot hold an answer, so when an output deny or redaction applies, the gateway answers the upgrade with 426 and Codex sends the session over HTTPS instead. One added while a socket is open closes it once the turn in flight ends, and Codex reopens it over HTTPS. On the socket an input denial reaches Codex as a 400 error event, as over HTTPS. While the config has a deny or redaction anywhere, a socket frame that is neither a request nor Codex's interrupt of its own response is refused the same way, since the checks cannot read it.

Credentials on pass-through traffic

A passthrough request is signed with the caller's own credential. From the provider's stored key it keeps the base URL and endpoint paths for every provider, plus these addressing fields:

Provider	Taken from the stored key	Never taken
bedrock	Region, the Claude over InvokeModel switches	Access key pair, session token, role, external id, auth type
azure-openai	Resource name, deployment id, API version	API key, AD token, auth mode, Entra, managed and workload identity fields
vertex-ai	Project id, region	Service account JSON
oracle	Region, compartment	API key, the IAM signing fields

Any other field on a stored key is never taken either.

On Bedrock the caller names its own credential: a Bedrock API key as the bearer with x-anyray-aws-auth-type: apiKey, or x-anyray-aws-access-key-id and x-anyray-aws-secret-access-key. A request with neither, or with x-anyray-aws-auth-type: assumedRole, gets a 400. The gateway would assume that role with its own AWS credentials.

On Azure, auth mode managed or workload gets a 400 on every route that reaches Azure OpenAI, because it uses the gateway's own Azure identity (Azure). The caller's API key, AD token or Entra client credentials work.

In pass-through, /v1/messages always uses the Anthropic seat lane or the request's custom host. The Bedrock and Azure rules apply to the other routes.

Pinning a target to a named provider key

Any target may carry provider_key_id naming one of the provider's named keys (the id only, never a credential):

{ "strategy": { "mode": "conditional",
    "conditions": [ { "query": { "metadata.team": "ml" }, "then": "ml-lane" } ],
    "default": "org" },
  "targets": [ { "name": "ml-lane", "provider": "openai", "provider_key_id": "team-ml" },
               { "name": "org", "provider": "openai" } ] }

Admin config only (ignored in a client x-anyray-config). Unknown ids are rejected on save; a later-deleted key fails pinned requests with 424, with no fallback.

Forwarded headers

A target's forward_headers list, or the x-anyray-forward-headers request header (comma separated), names request headers to copy upstream. The value always comes from the request.

A forwarded header never replaces the provider credential (Authorization, x-api-key, api-key) or a provider key's Custom headers. When the request runs on a key the gateway holds, it also never replaces any other header the gateway sets for that key: OpenAI-Organization and OpenAI-Project from the target's openai_organization and openai_project, Oracle's CompartmentId, or the signed headers of a SigV4 or OCI request. The gateway drops that copy and logs only the header's name, at debug level.

Account headers never forward beside a key the gateway holds, even when the gateway did not set one: OpenAI-Organization, OpenAI-Project, x-goog-user-project, x-goog-project-id, CompartmentId, and the x-anyray-* account headers.

On litellm, truefoundry and anyray-upstream, two configured values are not sent: a custom header the gateway drops (such as Connection), and a custom_body field the caller replaces with its own body or override_params. Such a value stops counting as a credential the gateway holds only when it appears nowhere else in the request (body, headers or URL). A common value such as Connection: keep-alive usually matches the client's own headers, so it still counts.

Still forwarded	Why
content-type, openai-beta, anthropic-beta, anthropic-version, x-grok-conv-id	The gateway builds them from the caller's request
Other headers the gateway did not set	There is nothing to replace

passthrough traffic, and a request served on the caller's own credentials, forward as before: the caller's copy wins over everything except the credential and Custom headers.

On a key the gateway holds, a caller's account fields in x-anyray-config (openai_organization, openai_project, resource_name, vertex_project_id, oracle_compartment_id, and the Bedrock role's aws_role_arn and aws_external_id) and the matching x-anyray-* headers are ignored. Account headers inside the caller's custom_headers are dropped too. The admin's routing target or provider key decides, else the key's default. On Vertex, a request on any route naming another project in its path or a $userProject / userProject parameter gets a 400 on a key the gateway holds.

A caller's auth_passthrough in x-anyray-config is ignored. Only x-anyray-auth-mode: passthrough selects pass-through.

Routes for people and agents

For API-key requests, precedence is x-anyray-config, then x-anyray-provider, then the signing person's or agent's route, then the org config. Own routes apply even when org routing is off. A delegating agent uses its own route, never the end user's.

A person route has a default, optional model rules, and optional providerKeys. Agents use their binding as the default and routingRules for model rules. Destinations are org, a single / loadbalance / fallback config, or subscription (people only). An exact model rule wins over a family rule; among families, the longest prefix wins. Otherwise the default applies. Model rules apply to JSON-bodied chat, messages, responses, and embeddings requests. Multipart uploads follow the route's default. Model matching normalizes provider prefixes and dated ids.

A key-signed request matching subscription uses the person's non-subscription default, else the org route, retaining their provider-key map. Pass-through traffic keeps its current provider and credential. /v1/me/models lists each destination's models only where the model rules send them there.

Agent bindings

Bind a provider and key on the agent itself (Agents page or at mint time; shown read-only under From Agents on Routing). Precedence: request x-anyray-config → x-anyray-provider → own route → org config. A request x-anyray-provider (or a model alias) picks the provider over both, but its key still comes from routing when routing names one account for that provider: the agent binding, else the org config (a lane on the model counts on chat, messages and new Responses requests). Otherwise the agent's own key for that provider serves, else the default key. Or write a lane:

{ "strategy": { "mode": "conditional",
    "conditions": [ { "query": { "metadata.agent": "release-bot" }, "then": "ci-lane" } ],
    "default": "org" },
  "targets": [ { "name": "ci-lane", "provider": "bedrock", "provider_key_id": "ci-account" },
               { "name": "org", "provider": "openai" } ] }

Only service keys match agent lanes.

Give an agent its own key for a provider

1. On the Routing page, under Agent keys, click Add.
2. Pick the agent, the provider, and that agent's named key.
3. Click Save. Add one row per agent and provider.

That agent's requests to that provider use its key, including simple model names such as a grok alias. Its other providers keep their default key:

Agent	Agent keys row	grok bills	claude bills
agent-a	x-ai → xai-agent-a	xai-agent-a	Anthropic default key
agent-b	x-ai → xai-agent-b	xai-agent-b	Anthropic default key

A key picked in Targeted routing, Split routing or Agent routing still wins. If the key is deleted, the agent's requests to that provider fail with 424 instead of billing another account. API field: routingProviderKeys (reference).

An agent's key goes to every host the org config sends that provider's default key to. So adding, changing or removing a row needs providerkeys:manage (providers:write for API keys) as well as clientkeys:manage. Without it the save returns 403 agent_provider_keys_requires_providerkeys_manage and nothing changes.

Targeted routing on the Routing page

Targeted routing adds lanes in front of the picked strategy, which becomes the default target (so conditional isn't in the picker).

Match	Condition it writes	Matched against
agent name	metadata.agent	The verified service key's name
agent id (survives rename)	metadata.agentId	The verified service key's id
user	metadata.user	The verified key-bound user
team	metadata.team	The verified key-bound team
model	params.model	The model named in the request

• Values match whatever their letter case. A $regex on the model also tests the id in lowercase, so ^grok- catches Grok-4.7. To leave a model out of a $regex lane, add a $nin test (it ignores letter case); a lookahead can still pass on the id as sent. Any other $regex matches as written.
• Name lanes stop matching on rename (traffic falls to the default); match on id instead. Revoked agents are flagged, not blocked.
• Hand-written conditions, a third test, or a $regex other than a model family lane show read-only; use PUT /admin/routing-config.
• A $regex lane lists every current priced model it matches in GET /v1/me/models.

Narrowing a lane with a second test

And also adds a second test on a different dimension (one query object = AND). Here Grok goes to each agent's own xAI key, everything else to a shared Bedrock account:

{ "strategy": { "mode": "conditional",
    "conditions": [
      { "query": { "metadata.agentId": "svc_a", "params.model": "grok-4.6" },
        "then": "agentId:svc_a+model:grok-4.6" },
      { "query": { "metadata.agentId": "svc_b", "params.model": "grok-4.6" },
        "then": "agentId:svc_b+model:grok-4.6" }
    ],
    "default": "default" },
  "targets": [
    { "name": "agentId:svc_a+model:grok-4.6", "provider": "x-ai", "provider_key_id": "xai-agent-a" },
    { "name": "agentId:svc_b+model:grok-4.6", "provider": "x-ai", "provider_key_id": "xai-agent-b" },
    { "name": "default", "provider": "bedrock", "provider_key_id": "shared" } ] }

Split routing writes this in one step: pick one or more agents, the models, a provider and a key, then Apply (one lane per agent and model, on agent id, above broader lanes). Applying again re-points instead of duplicating. The agents stay picked, so the next provider is one more Apply.

To send every model of a provider, pick All xAI models (or All OpenAI models, All Anthropic models; one per provider that holds a key). That writes one lane per agent, which also catches models released later, and the page lists the current models it covers:

{ "query": { "metadata.agentId": "svc_a", "params.model": { "$regex": "^grok-" } },
  "then": "agentId:svc_a+model~^grok-" }

A model picked by name for the same agent goes above its family lane, so grok-4.7 can bill another key while the other Grok models use the family lane's key.

Sending one person to their own provider

Users → person → Routing → provider (and key) → Save writes the same user lane. The row names earlier lanes that could win, and why it is read-only when it is (routing off, no default provider, hand-written config). Agents use their Agents binding instead.

Saving safely from two places

Send GET's revision back as expectedRevision; a stale one gets 409 with the winning revision.

Revision details

GET  /admin/v1/routing   →  { "config": {...}, "revision": "a1b2c3d4e5f60718" }
PUT  /admin/v1/routing      { "config": {...}, "expectedRevision": "a1b2c3d4e5f60718" }

Without expectedRevision, the save retries concurrent changes up to three times, resolving references and checking permissions against the latest config each time. Repeated races return 409 with a retry message. The revision is an opaque keyed token: key order and no-op saves do not change it. The console reloads on 409 and asks you to reapply.

Without a database, a routing file that exists but can't be read or parsed gets no revision. GET and any save with expectedRevision return 500 routing_config_unreadable and leave the file as it is. Restore the file or remove it (a missing file means no routing config), then reload.