OpenAI-compatible clients

Anything that targets the OpenAI API (the OpenAI SDKs, LangChain's ChatOpenAI, the Vercel AI SDK's OpenAI provider, LlamaIndex) points at http://<gateway>:8787/v1. Base URLs, client keys, and attribution headers are on SDKs & frameworks.

OpenAI (Python)

OpenAI (Python)

from openai import OpenAI

client = OpenAI(
    base_url="http://<gateway>:8787/v1",
    api_key="<your-gateway-key>",
    default_headers={
        "x-anyray-provider": "openai",
        "x-anyray-metadata": '{"user":"alice","team":"platform"}',
    },
)

client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": "ping"}],
)

OpenAI (TypeScript)

OpenAI (TypeScript)

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://<gateway>:8787/v1',
  apiKey: '<your-gateway-key>',
  defaultHeaders: {
    'x-anyray-provider': 'openai',
    'x-anyray-metadata': '{"user":"alice","team":"platform"}',
  },
});

LangChain

LangChain

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="http://<gateway>:8787/v1",
    api_key="<your-gateway-key>",
    model="gpt-5",
    default_headers={"x-anyray-provider": "openai"},
)

Vercel AI SDK

Vercel AI SDK

import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  baseURL: 'http://<gateway>:8787/v1',
  apiKey: '<your-gateway-key>',
  headers: { 'x-anyray-provider': 'openai' },
});

LlamaIndex

LlamaIndex

from llama_index.llms.openai import OpenAI

llm = OpenAI(
    api_base="http://<gateway>:8787/v1",
    api_key="<your-gateway-key>",
    model="gpt-5",
    default_headers={"x-anyray-provider": "openai"},
)

Running a LiteLLM proxy? Put the gateway in front of it; both topologies, front-door and in-process attach, are on the LiteLLM page.