API reference

The gateway's inference API plus its admin and inbound SCIM v2 surfaces.

Browse and try every endpoint — inference (/v1/*) and admin (/admin/*) — in the interactive API explorer, backed by the OpenAPI 3.1 spec (load it into Postman or codegen too). For a capability-annotated list of every route, see the Endpoint reference.

Open the API explorer

Base URL

The gateway is self-hosted — it runs on your own infrastructure (default port 8787), not on docs.anyray.ai. Put it behind TLS for any networked deployment.

Base URL
https://<your-anyray-gateway>        # e.g. http://localhost:8787 in-network

Deploying and configuring the gateway — provider keys, content mode, the optimizer URL, and the rest of its server-side settings — is covered in Configure.

Authentication

Send your Anyray client key as a bearer token, in x-anyray-api-key, or — for Anthropic-native clients on /v1/messages — in x-api-key. The real provider key stays server-side.

Authenticated request
curl https://<your-anyray-gateway>/v1/chat/completions \
  -H "Authorization: Bearer $ANYRAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H 'x-anyray-provider: openai' \
  -d '{
    "model": "gpt-4o",
    "messages": [{ "role": "user", "content": "Summarize the release notes." }]
  }'

There are four credentials:

Credential	Used on	Grants
Client key (`Authorization: Bearer`, `x-anyray-api-key`, or `ark_` in `x-api-key`)	`/v1/*` inference	The holder's user/team attribution. Minted by an admin enrollment flow or self-service SSO with `anyray-connect --sso`.
Service key (`ark_svc_…`, sent through the same headers)	`/v1/*` inference	A revocable non-human service identity for an AI agent, CI job, or SDK script, with an optional per-key monthly dollar budget. No interactive SSO.
Admin token (`ANYRAY_ADMIN_TOKEN`)	The console + `/admin/*` config API	Break-glass `owner` access, kept server-side. Each `/admin/*` route requires a capability; enterprises can add console SSO + RBAC.
SCIM bearer (`Authorization: Bearer`)	`/scim/v2/*`	Lets the configured IdP provision users and groups for one tenant. Only its SHA-256 hash is stored.

A minted client key is always required

Every /v1/* request needs a valid minted key. A developer can obtain one without an invite by running the SSO enrollment command from Users → SSO enrollment after the organization configures SSO. Missing, unknown, expired, revoked, and SCIM-deactivated identities return HTTP 401 with missing_key, key_unknown, key_expired, key_revoked, or user_deactivated and a safe fix hint. The key's user and team control attribution; x-anyray-metadata cannot override them. An org admin can also mint and rotate keys. OAuth/OIDC never runs on /v1/* itself.

Service keys for agents and CI

The admin API manages service keys through three routes:

Method	Path	Capability	Shape
post	`/admin/service-keys`	`clientkeys:manage`	`{ "name": string, "team": string, "monthlyBudgetUsd"?: number }` → `201 { "key": "ark_svc_…", "record": {...} }`
get	`/admin/service-keys`	`config:read`	`{ "keys": [...] }`; metadata only, filtered to service keys, with `source` and `type` set to `service`
delete	`/admin/service-keys/:id`	`clientkeys:manage`	Revokes the key and returns `{ "ok": true }`

Mint a budgeted CI key
curl https://<your-anyray-gateway>/admin/service-keys \
  -H "Authorization: Bearer $ANYRAY_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "synthetic-ci-release",
    "team": "platform",
    "monthlyBudgetUsd": 25
  }'

Store the returned key immediately; the gateway stores only its SHA-256 hash and never returns the raw token again. A budget counts successful request cost per UTC calendar month. At or above the budget, covered inference routes return:

HTTP 402
{
  "error": {
    "code": "service_key_budget_exceeded",
    "message": "This service key has reached its monthly budget. Ask an administrator to raise the budget or wait for the next UTC month."
  }
}

This is a soft, fail-open cap. Completed requests update the shared counter after their responses, so concurrent requests can overshoot; a counter outage lets requests proceed. A key without monthlyBudgetUsd, or with a value of 0, is unlimited.

Request headers

Beyond Authorization / Content-Type, the gateway reads a few x-anyray-* headers:

x-anyray-providerstring

Provider to route to — openai, anthropic, vertex-ai, bedrock, azure-openai, … Optional when a default provider or routing config is set server-side.

x-anyray-metadatastring (JSON)

Attribution, e.g. {"user":"alice","team":"platform"} — drives spend attribution and per-user caps. Never carries prompt/response content.

x-anyray-configstring (JSON)

Per-request routing — fallback list, load-balance weights, retry policy. No credentials; they resolve from the server-held key store.

Response headers

Every response carries x-anyray-provider (the provider the request was routed to) and x-anyray-trace-id. When the provider returns a server error (any 5xx — e.g. an Anthropic 529 overloaded_error), the gateway also stamps the markers below so you can tell an upstream outage apart from a gateway fault. The upstream error body is forwarded unchanged.

x-anyray-error-sourcestring

Present and set to upstream when the error originated at the provider, not the gateway. Absent on success and on gateway-generated errors.

x-anyray-upstream-providerstring

The upstream provider that returned the error — anthropic, openai, vertex-ai, … (omitted when the provider is unknown).

x-anyray-upstream-statusstring

The provider's own HTTP status (e.g. 529, 503).

Telling provider errors from gateway errors

No x-anyray-error-source: upstream header on a 5xx means the gateway itself failed — check your deployment. With it, the fault is on the provider's side (capacity/overload); retrying shortly usually clears it.

Endpoints

The gateway serves an OpenAI-compatible inference API and the Anthropic-native Messages API on /v1/*, the admin-gated configuration & reporting API on /admin/*, the pre-key /sso/cli/* self-service login handshake used by Connect, and an inbound SCIM 2.0 server on /scim/v2/*. The Endpoint reference is the complete, capability-annotated index of every route, and full request/response schemas live in the OpenAPI spec. The optimizer's own before/after-request hooks are the separate optimizer protocol.

Not every endpoint runs on every provider

Chat completions, embeddings, and the Anthropic Messages API work across all supported providers. Image, audio, Responses, file, and Batch endpoints are OpenAI / Azure-only passthrough (audio also works on groq).

SCIM v2 provisioning

Point the IdP at this base URL and send the separately configured SCIM bearer on every request:

SCIM base URL
https://<your-anyray-gateway>/scim/v2

Resource	Routes
Discovery	`GET /ServiceProviderConfig`, `GET /ResourceTypes`, `GET /Schemas`
Users	`POST /Users`, `GET /Users`, `GET /Users/:id`, `PUT /Users/:id`, `PATCH /Users/:id`, `DELETE /Users/:id`
Groups	`POST /Groups`, `GET /Groups`, `GET /Groups/:id`, `PUT /Groups/:id`, `PATCH /Groups/:id`, `DELETE /Groups/:id`

User lists support filter=userName eq "dev@example.com", startIndex, and count. Group lists support the same paging fields. User PATCH accepts active:false for immediate offboarding; Group PATCH accepts add/remove/replace member operations. A missing group member can be created when the member carries a verified email in display or value.

Configure the bearer, admin group, and team mappings through the deployment-owner admin lane:

Configure SCIM
curl -X PUT https://<your-anyray-gateway>/admin/scim/settings \
  -H "Authorization: Bearer $ANYRAY_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bearerToken": "synthetic-scim-bearer-token-replace-me-0001",
    "adminGroup": "Anyray Admins",
    "groupTeamMap": { "Engineering": "platform" }
  }'

GET /admin/scim/settings returns configured, adminGroup, groupTeamMap, and updatedAt. It never returns the bearer or bearer hash. A later PUT can omit bearerToken to retain the current credential.

SCIM errors always use application/scim+json and this envelope:

SCIM error
{
  "schemas": ["urn:ietf:params:scim:api:messages:2.0:Error"],
  "detail": "invalid bearer token",
  "status": "401"
}

Errors

Most inference errors use the OpenAI envelope ({ "error": { "message", "type", "code" } }). Client-key authentication uses the fixed content-free machine envelope shown below.

Status	Meaning
`400`	Malformed request — failed validation, bad JSON, or an unknown provider/model.
`401`	Missing, malformed, unknown, expired, revoked, or SCIM-deactivated client identity. The body includes `status: failure`, a reason `code`, the fixed `valid client key required` message, and a safe `hint`.
`402`	Deployment entitlement/spend cap, or a service key's monthly dollar budget, has been reached. Service-key budget responses use `error.code: service_key_budget_exceeded`.
`429`	Rate limit or per-user monthly token cap exceeded. Honors `Retry-After`.

Base URL​

Authentication​

Service keys for agents and CI​

Request headers​

Response headers​

Endpoints​

SCIM v2 provisioning​

Errors​