Provider keys

Provider credentials live server-side and rotate at runtime, without a redeploy.

Set a key with anyray-connect admin secret set provider_key <slug>, through your agent's admin MCP, on the Providers page, or with GET / PUT /admin/provider-keys. This runtime path wins over the ANYRAY_PROVIDER_KEY_* env vars, and keys stay server-side. With no key set, BYO clients must send both the provider credential and their Anyray client key. GET reads the shared store and answers 503 if it does not reply within 3 seconds, never an older copy. Retry it.

If this gateway cannot decrypt the stored keys (its ANYRAY_ADMIN_TOKEN differs from the one that sealed them), GET answers an empty config with unreadable: true, and so does /admin/provider-capabilities. A save there replaces every stored key, for every replica, so the write answers 409 provider_store_unreadable unless it carries ?replaceUnreadableStore=true. The Providers page warns and asks you to confirm first. Clears and per-key edits (field patches included) answer 503 provider_store_unreadable_edit, even with the confirm. After that answer the page asks for the same confirm on Save, which re-enters the provider's keys. Remove and Make default stay disabled until the keys can be read again. Restoring the token that sealed the keys keeps them all. An invalid patch answers 400; no_provider_named means every slug in the request is empty. Writes answer 503 if the store cannot be read or does not reply.

The gateway does not forward an Anyray client key (ark_…) as a provider credential. Outside pass-through, a request whose only credential is that key gets 424 no provider key configured for <provider> whenever the provider's auth mode would send it: Bedrock with x-anyray-aws-auth-type: apiKey, Vertex AI without a service account, Azure OpenAI on an API key or AD token, and Oracle without IAM signing. On /v1/realtime the gateway refuses the upgrade instead of dialing. Modes that sign or mint their own credential (AWS keys or a role, a service account, Entra, managed or workload identity, Oracle signing) ignore the client key and serve the request.

A new node-written outbound origin needs providerkeys:manage (providers:write for API keys), in addition to routing:write, even on a node without a provider. A new pairing of a credential with a node-written base URL (including its path) or custom_endpoints also needs that grant. A host injected from the node's own provider key is key-sourced and needs no extra grant. Copying that key's Base URL into routing is node-written, even when the URLs match. An inherited host is node-sourced for the child credential.

The comparison covers roots, groups and all branches, regardless of strategy. Credential selectors distinguish named pins, inline bags and unpinned resolution. Inline bags and headers use separate keyed HMACs. Every inline credential field counts, including beside a pin; changing or removing one needs a new grant. Header changes on node-sourced hosts need a new grant. The MACs stay inside the save comparison and need no private admin token. Save checks read provider keys from authority; an unavailable authority returns 503 provider_keys_authority_unavailable. Operators can reorder, move, duplicate or remove existing grants. A new grant without permission records the actor, code and route in the admin audit and returns 403 outbound_host_requires_providerkeys_manage.

Named provider keys

A provider can hold several named keys. The default key serves every request that doesn't pin an id. On the Providers page, Add key takes a lowercase id (such as team-ml), an optional label, and the credential. On a provider with no credential yet, the first named key becomes the default.

PUT /admin/provider-keys returns {status, capabilities} with provider slugs and key ids only, never a credential. It patches:

Patch	Effect
{ "keys": { "<id>": <credential> } }	Adds or replaces that named key.
{ "keys": { "<id>": { "fields": { "<field>": "<value>" } } } }	Changes only the named fields of an existing key. An omitted field keeps its stored value; null or "" removes one.
"" or null under an id	Removes that key.
"defaultKeyId": "<id>"	Which key serves unpinned requests.
A bare credential for the provider	Replaces the default key.
"" for the provider	Removes all of its keys.

The Providers page saves changed fields this way when capabilities.fieldPatches is true. Remove with "" or null. A key, bag field or field value that is only whitespace, and a bag with no non-empty value ({} included), are refused with 400 empty_provider_key and nothing is saved. Two field names that match once trimmed are refused with 400 too.

Pin a key from a routing target with provider_key_id (Routing). Each spend row records the key id that served it (providerKeyId). On the Routing page each target and lane has a Key column: a picker once the provider holds two or more keys, otherwise default key.