SDKs & frameworks

Any OpenAI- or Anthropic-based SDK routes through Anyray via its base URL.

What's specific

Property	Value
Type	Client library / framework (points at the gateway)
What changes	The client's base URL (+ optional provider header)
OpenAI-compatible base URL	`http://<gateway>:8787/v1`
Anthropic-native base URL	`http://<gateway>:8787`
Credential	Your gateway key (per user), not the provider key
Attribution	`x-anyray-metadata` JSON header — `{"user":"…","team":"…"}`

Everything shared — self-hosting, how the gateway reaches providers, where credentials are stored — is documented once in the gateway. For the raw HTTP surface, see the API reference.

Keys and the provider header

Pass your gateway key as the SDK's API key — the real provider key stays server-side. Get one by enrolling with anyray-connect --enroll; a bare placeholder is rejected when the gateway requires verified users. The x-anyray-provider header is optional when a default provider or routing config is set server-side.

OpenAI-compatible clients

Anything that targets the OpenAI API — the OpenAI SDKs, LangChain's ChatOpenAI, the Vercel AI SDK's OpenAI provider, LlamaIndex — points at http://<gateway>:8787/v1.

OpenAI (Python)
OpenAI (TypeScript)
LangChain
Vercel AI SDK
LlamaIndex

OpenAI Python SDK
from openai import OpenAI

client = OpenAI(
    base_url="http://<gateway>:8787/v1",
    api_key="<your-gateway-key>",
    default_headers={
        "x-anyray-provider": "openai",
        "x-anyray-metadata": '{"user":"alice","team":"platform"}',
    },
)

client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": "ping"}],
)

OpenAI Node SDK
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://<gateway>:8787/v1',
  apiKey: '<your-gateway-key>',
  defaultHeaders: {
    'x-anyray-provider': 'openai',
    'x-anyray-metadata': '{"user":"alice","team":"platform"}',
  },
});

langchain-openai
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="http://<gateway>:8787/v1",
    api_key="<your-gateway-key>",
    model="gpt-5",
    default_headers={"x-anyray-provider": "openai"},
)

@ai-sdk/openai
import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  baseURL: 'http://<gateway>:8787/v1',
  apiKey: '<your-gateway-key>',
  headers: { 'x-anyray-provider': 'openai' },
});

llama-index
from llama_index.llms.openai import OpenAI

llm = OpenAI(
    api_base="http://<gateway>:8787/v1",
    api_key="<your-gateway-key>",
    model="gpt-5",
    default_headers={"x-anyray-provider": "openai"},
)

Anthropic-native clients

The Anthropic SDKs and LangChain's ChatAnthropic speak the Messages API. Point them at http://<gateway>:8787 — no /v1 suffix, the SDK appends /v1/messages itself.

Anthropic (Python)
Anthropic (TypeScript)
LangChain

Anthropic Python SDK
from anthropic import Anthropic

client = Anthropic(
    base_url="http://<gateway>:8787",
    api_key="<your-gateway-key>",
    default_headers={"x-anyray-provider": "anthropic"},
)

client.messages.create(
    model="claude-sonnet-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "ping"}],
)

@anthropic-ai/sdk
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  baseURL: 'http://<gateway>:8787',
  apiKey: '<your-gateway-key>',
  defaultHeaders: { 'x-anyray-provider': 'anthropic' },
});

langchain-anthropic
from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    base_url="http://<gateway>:8787",
    api_key="<your-gateway-key>",
    model="claude-sonnet-5",
    default_headers={"x-anyray-provider": "anthropic"},
)

LiteLLM

LiteLLM can route to the gateway as an OpenAI-compatible upstream (set api_base to http://<gateway>:8787/v1), or run the optimizer in-process via the BYO attach mode. Both paths are covered in the gateway.

Close the retrieval loop in an agent SDK

A base URL is enough to route a shell command or SDK request, but it is not a retrieval path. A plain headless script cannot see an Anyray ctx_… handle and call the model-facing retrieval tool on its own. The gateway therefore keeps retrieval-dependent strategies disabled until it receives real capability evidence; a base URL, API key, or SDK user-agent never opts a client in by itself.

If the SDK hosts an agent that supports stdio MCP, register the read path in that agent. For the Claude Agent SDK's TypeScript query options, the complete server entry is one line:

Claude Agent SDK options
mcpServers: { anyray: { type: 'stdio', command: 'anyray-connect', args: ['__anyray-mcp-server', 'profile'] } }

This mounts the exact stdio command anyray-connect __anyray-mcp-server profile. The executable must be a durable Connect install on the agent process's PATH, with an enrolled profile for the gateway that receives the model requests. On MCP initialization, the server exposes anyray_retrieve and anyray_recall; the agent can then read a handle back, and the gateway receives real capability evidence. That evidence unlocks the full reversible strategy set for the enrolled key while it remains fresh.

Registering the server is the opt-in. Starting it beside a client that cannot expose MCP tools to the model does not close the loop. There is no retrieval-capability environment variable and no manual flag to assert.

Coding tools & shell

For coding tools (Claude Code, Cursor, Codex) and a shell/SDK environment, let anyray-connect write the base URL and enroll a per-user key for you — one command points your tools at the gateway. See also the Cursor integration.

What's specific​

OpenAI-compatible clients​

Anthropic-native clients​

LiteLLM​

Close the retrieval loop in an agent SDK​

Coding tools & shell​