SDKs

Change the base URL, use a client key, and your SDK calls route through the gateway. No client library and no code changes.

1

Mint a client key

An agent key for a script or service, or a seat key for your own machine. Console → Users → Invite users → Agents, or the CLI:

anyray-connect enroll --agent nightly-batch

The key looks like ark_…. Detail: Client keys.

2

Set two environment variables

export ANYRAY_GATEWAY_URL="https://gateway.example.com"
export ANYRAY_CLIENT_KEY="ark_synthetic-123456"

3

Point the SDK at the gateway

Pick the API your code already speaks:

OpenAI Python

OpenAI Python

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['ANYRAY_GATEWAY_URL']}/v1",
    api_key=os.environ["ANYRAY_CLIENT_KEY"],
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Ping"}],
)

OpenAI TypeScript

OpenAI TypeScript

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: `${process.env.ANYRAY_GATEWAY_URL}/v1`,
  apiKey: process.env.ANYRAY_CLIENT_KEY,
});

const resp = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{role: 'user', content: 'Ping'}],
});

Anthropic Python

Anthropic Python

import os
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ["ANYRAY_GATEWAY_URL"],
    api_key=os.environ["ANYRAY_CLIENT_KEY"],
)

msg = client.messages.create(
    model="claude-opus-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "Ping"}],
)

Anthropic TypeScript

Anthropic TypeScript

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  baseURL: process.env.ANYRAY_GATEWAY_URL,
  apiKey: process.env.ANYRAY_CLIENT_KEY,
});

const msg = await client.messages.create({
  model: 'claude-opus-5',
  max_tokens: 256,
  messages: [{role: 'user', content: 'Ping'}],
});

Note the /v1 on the OpenAI lane and its absence on the Anthropic one. The SDKs append different paths, so both examples reach the same gateway.

4

Attribute the traffic

A key carries its own identity, so this is optional. Send it when one key serves several people, or when you want a job name in the console.

-H 'x-anyray-metadata: {"user":"batch@example.com","team":"data"}'

Every SDK has a default-headers option. The gateway trusts the header only on a delegated key. On a per-user key it keeps the key's own identity.

5

Confirm it routed

Open the console Traces page, or read the response headers. Every gateway response carries x-anyray-provider and x-anyray-request-id. A request that never reached the gateway has neither.

Which endpoint your SDK gets

You do not choose a lane. The gateway reads the path.

Your SDK calls	Base URL to set	Gateway serves
/v1/chat/completions, /v1/responses, /v1/embeddings	$ANYRAY_GATEWAY_URL/v1	The OpenAI-compatible lane
/v1/messages	$ANYRAY_GATEWAY_URL	The Anthropic-native lane

Both lanes optimize, meter, and attribute the same way. Streaming, tool calls, and structured outputs pass through unchanged.

Anything that speaks either API works the same way: LangChain, LlamaIndex, Vercel AI SDK, LiteLLM, Instructor, curl. Set the base URL and the key, and skip any per-provider setup the library offers.

Client keys

Kind	For	Mint it with
Seat key	A person's own machine	anyray-connect --enroll or --sso
Agent key	A script, service, or CI job	anyray-connect enroll --agent <name>
Delegated key	One backend serving many users	Console → Users → Invite users → Agents

Store it the way you store any secret. It is a gateway credential, not a provider key, and the gateway holds the provider keys for you.

Rotation, revocation, and expiry: Key renewal.

Your code sends a provider key today

Replace it with the client key and change the base URL. Nothing else moves. Leave the provider key in place if you want a fallback, because the gateway ignores it.

Never ship a provider key to a machine you do not control. Put the provider keys on the gateway (Provider keys) and give each client an ark_… key you can revoke on its own.

Choosing the model and the provider

Pass the model id your provider uses. The gateway routes it by routing rules, so the same id can serve from a different provider without a code change.

To pin one request to a provider, send x-anyray-provider: openai. Use it for a test, not as a default, because a pin skips routing.

If your agent calls tools

The optimizer trims bulky content and leaves a short ctx_… handle behind. Your agent needs a way to read the original back, or the model works from a hole. The gateway serves that read path itself, so there is nothing to install.

A workflow of one-shot calls can skip this. A pipeline that cannot call tools runs the no-retrieve lane by itself, and nothing is elided.

1

Register the gateway as an MCP server

Use your gateway address plus /mcp, and the same client key. If your base URL ends in /v1, drop that part, because /v1/mcp goes upstream as a chat request and fails with a 500.

Claude Agent SDK (Python)

mcp_servers={
    "anyray": {
        "type": "http",
        "url": "https://gateway.example.com/mcp",
        "headers": {"x-anyray-api-key": key},
    }
}

2

Allow the two tools

Agent hosts prefix MCP tool names with the server name. If yours has an allowlist, add both, or the model is offered them and refused on every call.

allowed_tools=[*existing_tools, "mcp__anyray__anyray_retrieve", "mcp__anyray__anyray_recall"]

Hosts without an allowlist skip this step.

3

Run the agent once, then check the console

Any authenticated call proves the path. Console → Users → Agents: the key reads Active. Active · retrieval not seen means the agent reaches the gateway but the read path is unproven, so retrieval-dependent optimization stays off.

Hosted connectors, a plain-HTTP tool loop without MCP, and the stdio server: Retrieval reference.

Troubleshooting

What you see	Cause	Fix
401	The key is wrong, revoked, or expired	Mint a new one, then check Key renewal
404 on a path you expect	The /v1 suffix is on the wrong lane	See Which endpoint your SDK gets
The call works, but nothing lands in the console	The SDK never used your base URL	Check the response for x-anyray-request-id
The right user never appears	The key is per-user, so the header is ignored	Expected. Use a delegated key to attribute per request

More: Troubleshooting.

Setting up a coding tool rather than a script? Let anyray-connect write the base URL and enroll a key for you.

Register the source hook together with MCP, before appending tool results to the transcript. Use the complete tool-loop options; the Agents row polls until source trim and retrieval are seen.