SDKs

Change the base URL and use a client key. No client library, no code changes. Prompt caching and reading back trimmed content run on the gateway.

Mint a client key

Console → Users → Agents → Add agent. You get an ark_… key (kinds) that already carries the user and team.

Set two environment variables

export ANYRAY_GATEWAY_URL="https://gateway.example.com"
export ANYRAY_CLIENT_KEY="ark_synthetic-123456"

Hand the prompt to your coding agent

In the Add agent dialog, open the OpenAI or Anthropic tab and copy its prompt into the coding agent working on your app. The code it writes reads the two variables above; never paste the key into the chat. The prompt ends by polling /connect/self-check until problems is 0.

The prompts are also at https://app.anyray.ai/agents/openai.md and https://app.anyray.ai/agents/anthropic.md; the agent asks you for the gateway URL.

Tool-output trimming

The prompt always wires the Anyray MCP server. Whether the gateway trims tool output in your app's tool loop is the gateway setting harnessRetrieval, on by default: the gateway may trim large tool output behind handles the model reads back. Turning it off keeps retrieval-dependent trims off; lossless folds and cache optimization can still run. An existing request override takes precedence. An app that reads local files and takes input only from trusted people gets the anyray-connect stdio server, which adds batch reads (anyray_read_batch). An app that fetches web pages gets the gateway's endpoint, because fetched content is untrusted input.

On the Anthropic tab, a Claude Agent SDK app also sets settingSources: [] and installs the anyray skill in its own plugin. A plain Anthropic client has no skill loader and skips that. Its stdio server runs as anyray-connect __anyray-mcp-server sdk (what it lists).

Or point the SDK at the gateway by hand

OpenAI Python
OpenAI TypeScript
Anthropic Python
Anthropic TypeScript

OpenAI Python
import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['ANYRAY_GATEWAY_URL']}/v1",
    api_key=os.environ["ANYRAY_CLIENT_KEY"],
)

resp = client.chat.completions.create(
    model="gpt-6-luna",
    messages=[{"role": "user", "content": "Ping"}],
)

OpenAI TypeScript
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: `${process.env.ANYRAY_GATEWAY_URL}/v1`,
  apiKey: process.env.ANYRAY_CLIENT_KEY,
});

const resp = await client.chat.completions.create({
  model: 'gpt-6-luna',
  messages: [{role: 'user', content: 'Ping'}],
});

Anthropic Python
import os
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ["ANYRAY_GATEWAY_URL"],
    api_key=os.environ["ANYRAY_CLIENT_KEY"],
)

msg = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "Ping"}],
)

Anthropic TypeScript
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  baseURL: process.env.ANYRAY_GATEWAY_URL,
  apiKey: process.env.ANYRAY_CLIENT_KEY,
});

const msg = await client.messages.create({
  model: 'claude-opus-5-5',
  max_tokens: 256,
  messages: [{role: 'user', content: 'Ping'}],
});

The OpenAI lane takes /v1; the Anthropic lane does not.

Attribute the traffic (optional)

Only a delegated key's header is trusted; other keys carry their own identity.

-H 'x-anyray-metadata: {"user":"batch@example.com","team":"data"}'

Add sessionId when a multi-step agent makes several calls. Name the step, not the run. Without it, each call is its own row on the Sessions page.

-H 'x-anyray-metadata: {"user":"batch@example.com","sessionId":"billing-agent.planner"}'

Use a fixed name per node, the same on every run. On OpenAI models it also feeds the prompt-cache key, so a per-run id breaks cache hits (details).

Confirm it routed

Console Sessions, or the x-anyray-provider and x-anyray-request-id response headers.

Which endpoint your SDK gets

The gateway picks the lane from the path. Streaming, tools, and structured outputs pass through.

Your SDK calls	Base URL to set	Gateway serves
`/v1/chat/completions`, `/v1/responses`, `/v1/embeddings`	`$ANYRAY_GATEWAY_URL/v1`	The OpenAI-compatible lane
`/v1/messages`	`$ANYRAY_GATEWAY_URL`	The Anthropic-native lane
`/v1/messages/batches`	`$ANYRAY_GATEWAY_URL`	Anthropic Message Batches
`/v1/batches`	`$ANYRAY_GATEWAY_URL/v1`	The OpenAI-compatible Batch API

Frameworks take the same two values: LangChain and LangGraph, and the agent SDKs and others.

Batch only work nobody is waiting on

Half price, unstreamed, within 24 hours (usually one). Good for backfills and evals. Never batch a conversation: a late turn loses its prompt cache and costs more than the discount saves.

Client keys

Kind	For	Mint it with
Seat key	A person's own machine	`anyray-connect --enroll` or `--sso`
Agent key	A script, service, or CI job	Console Add agent, or `anyray-connect enroll --agent <name>`
Delegated key	One backend serving many users	Admin API with `delegatedAttribution: true` (mint)

Rotation and expiry: Key renewal.

Your code sends a provider key today

Swap it for the client key; the gateway ignores provider keys. Keep those on the gateway (Provider keys).

Choosing the model and the provider

Pass your provider's model id; routing rules pick the provider. x-anyray-provider: openai pins one request and skips routing (tests only).

If your agent calls tools

A tool loop needs no extra setup for transforms that keep their values inline. Without a read path, retrieval-dependent trims stay off. The console calls this Active · forwarded unchanged; other configured transforms can still run. For verbatim input, use the exact profile.

An admin can turn on gateway-served retrieval for these loops by adding no_discovery to servedRetrievalClasses. The gateway then trims tool results behind a ctx_… handle, offers the model anyray_retrieve and anyray_recall, and answers those calls itself. It is off by default: on a measured tool loop, serving retrieval cost more than forwarding unchanged.

On a deployment that turned it on, register the gateway's MCP server only when a request falls outside it:

Your requests	Why the gateway cannot serve retrieval
Extended thinking on `/v1/messages`	A thinking block's signature binds to rounds your client never sees
`/v1/responses` without `store: false`	That lane is served only with `store: false`
A `tool_choice` other than `auto`	The model may not pick the retrieval tool

Rarer cases (passthrough, subscription keys, n above 1) are in the full list.

For those, add /mcp on the gateway origin (/v1/mcp fails with 500) to every model route the agent uses, and allow its tools:

Claude Agent SDK (Python)
mcp_servers={
    "anyray": {
        "type": "http",
        "url": "https://gateway.example.com/mcp",
        "headers": {"x-anyray-api-key": key},
    },
}
allowed_tools=[*existing_tools, "mcp__anyray__anyray_retrieve", "mcp__anyray__anyray_recall"]

Hosted connectors, org connectors (/mcp/org), and plain-HTTP loops: Agent tool-loop reference.

For dependent read tools in a TypeScript loop, see the optional read-only workflow helper.

Troubleshooting

What you see	Cause	Fix
`401`	The key is wrong, revoked, or expired	Mint a new one, then check Key renewal
`404` on a path you expect	The `/v1` suffix is on the wrong lane	See Which endpoint your SDK gets
The call works, but nothing lands in the console	The SDK never used your base URL	Check the response for `x-anyray-request-id`
The right user never appears	The key is per-user, so the header is ignored	Expected. Use a delegated key to attribute per request

More: Troubleshooting. Coding tools: use anyray-connect.

Which endpoint your SDK gets​

Client keys​

If your agent calls tools​

Troubleshooting​

Which endpoint your SDK gets

Client keys

If your agent calls tools

Troubleshooting