Skip to main content

Retrieval reference

Close the retrieval loop in an agent SDK​

When the optimizer elides bulk it leaves a ctx_… handle in the prompt, and the model needs a way to read the original back. The gateway keeps retrieval-dependent strategies disabled until it receives two things:

FactWhat proves it
A callable retrieval pathA declared anyray_retrieve tool, or a registered MCP server.
Fresh authenticated livenessAuthenticated MCP, a heartbeat, or model retrieval activity.

Nothing else opts a client in. Not a base URL, an API key, or an SDK user-agent. Starting a server beside a client that cannot expose MCP tools to the model does not close the loop, and neither does implementing POST /connect/retrieve without declaring a tool the model can call. There is no retrieval-capability environment variable and no manual flag.

The simplest opt-in is that the gateway is itself a remote MCP server. POST /mcp (streamable HTTP) serves anyray_retrieve and anyray_recall, gated by the same client key the agent already sends. Any MCP-capable runtime closes the loop with one entry:

Claude Agent SDK query options
mcpServers: {
  anyray: {
    type: 'http',
    url: 'http://<gateway>:8787/mcp',
    headers: { 'x-anyray-api-key': '<your-gateway-key>' },
  },
}

OpenClaw's equivalent entry is in its guide.

The path is /mcp, not /v1/mcp

/v1/* is the inference proxy and catches every path it does not recognize, so a request to /v1/mcp goes upstream as an inference call and returns 500 rather than 404.

A workflow of one-shot calls needs none of this. A summarize, extract, write pipeline, a batch job, or an embeddings-only service cannot call tools, so it runs the no-retrieve lane automatically and nothing is elided behind a handle the flow could not resolve.

The gateway MCP endpoint​

POST /mcp speaks the stateless streamable-HTTP transport: one JSON-RPC message per POST, notifications acknowledged with 202, and no session id. Every valid authenticated message renews the key's 6-hour retrieval lease.

The host that sent the handshake is the process exposing tools to the model, so retrieval-dependent optimization unlocks after initialization without waiting for a retrieval call. The endpoint advertises the retrieval pair only, because the stdio server's local-workspace tools (anyray_search, anyray_read) act on files a remote endpoint does not have.

Hosted connectors​

Zero client-side runtime: the platform calls your gateway's /mcp from the outside.

Anthropic's Messages API needs all three parts. mcp_servers, its paired mcp_toolset entry in tools (a server entry without one is a validation error), and the beta header:

Request params (header: anthropic-beta: mcp-client-2025-11-20)
"mcp_servers": [
  { "type": "url", "url": "https://<gateway>/mcp", "name": "anyray",
    "authorization_token": "<your-gateway-key>" }
],
"tools": [ { "type": "mcp_toolset", "mcp_server_name": "anyray" } ]

Or the OpenAI Responses hosted tool, inside tools:

{ "type": "mcp", "server_label": "anyray", "server_url": "https://<gateway>/mcp",
  "headers": { "x-anyray-api-key": "<your-gateway-key>" }, "require_approval": "never" }
ConstraintDetail
Same public URLUse the gateway URL the SDK already sends requests to. The gateway credits a connector as the read path only when it targets its own /mcp.
Publicly reachableThe platform calls from the outside, so a VPC-private gateway cannot use hosted connectors. Register the MCP server in the agent runtime instead.
First-party APIs onlyPer the providers' own compatibility notes, Anthropic's connector is not available on Amazon Bedrock or Google Cloud.
OpenAI: Responses API onlyOn chat.completions, declare and implement the tool yourself, or register the MCP server in your agent runtime.

A deployment routing Claude through Bedrock or Vertex loses nothing else. The declared-tool path and the runtime MCP entry are provider-independent: the marker rides the prompt, the tool rides the request, and the agent's own loop calls the gateway directly. The declaration makes the tool selectable, and authenticated MCP activity supplies the live lease.

Plain HTTP tools (your own tool loop, no MCP)​

For an agent that runs its own tool loop with no MCP support. Three independent things must be true:

1
The declaration
Proves the model can select the tool on this turn. It matches on the tool name (anyray_retrieve, or an mcp__…__anyray_retrieve-style namespaced form), so the name is load-bearing and must not be aliased.
2
The implementation
Against POST /connect/retrieve. This is what makes the handle resolvable once the model calls it.
3
The liveness signal
Proves the implementation is running. Send an authenticated empty POST /connect/mcp-heartbeat when the worker starts, and every 5 minutes while it can execute tool calls. Stop heartbeats before shutting the executor down. The lease expires after 6 hours without verified activity.

A declaration qualifies for that turn. The lease covers later turns whose tool catalog is deferred.

The declaration shape follows whichever dialect you already call. Both satisfy the gateway:

Anthropic dialect
ANYRAY_RETRIEVE_TOOL = {
    "name": "anyray_retrieve",
    "description": (
        "Fetch the full original text behind an Anyray handle. When a message "
        "contains a marker like [anyray: ... retrieve ctx_abc123], pass that "
        "ctx_ handle here to read what it replaced."
    ),
    "input_schema": {
        "type": "object",
        "properties": {"handle": {"type": "string"}},
        "required": ["handle"],
    },
}

The description is doing real work. The gateway unlocks retrieval on the name, but the model only calls the tool if it knows what a ctx_… marker is. Then implement it, in either dialect:

Implement
def anyray_retrieve(handle: str) -> str:
    r = requests.post(
        "http://<gateway>:8787/connect/retrieve",
        headers={"x-anyray-api-key": "<your-gateway-key>"},
        json={"handle": handle},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()["content"]

The response is {"status":"ok","content":"…","handle":"ctx_…"}. startLine, endLine, and grep are optional body fields for reading back a slice, and an unknown or expired handle answers 404. The stdio server below wraps this same route, so neither path is more capable than the other.

Stdio MCP (Connect already installed)​

Where anyray-connect is installed, its stdio server is equivalent to the remote endpoint, being a wrapper over the same routes, and also serves the local-workspace tools anyray_search and anyray_read. For the Claude Agent SDK's TypeScript query options, the complete server entry is one line:

Claude Agent SDK options
mcpServers: { anyray: { type: 'stdio', command: 'anyray-connect', args: ['__anyray-mcp-server', 'profile'] } }

This mounts the exact stdio command anyray-connect __anyray-mcp-server profile. The executable must be a durable Connect install on the agent process's PATH, with an enrolled profile for the gateway that receives the model requests. On MCP initialization the server exposes anyray_retrieve and anyray_recall. Connect renews that evidence every 5 minutes while the initialized stdio transport is alive, and the gateway expires it 6 hours after the last accepted heartbeat or model retrieval call.

How the model knows when to call it​

You do not need to teach the happy path. Every elision leaves a marker in-band, at the spot the content was, and the marker names the verb:

[anyray: observation masked · 330509 chars · retrieve ctx_9f3a2b via anyray_retrieve]
Do not rewrite the marker or the tool description

This wording is load-bearing and measured. Before the marker named the tool, the fleet resolved 0.4% of emitted handles. Naming anyray_retrieve outright is what made models call it.

Reinforce it with a system-prompt line

This is the only teaching channel that reaches every runtime, and the main way to surface anyray_recall, which nothing in-band points at once a marker has scrolled out of view or been compacted away.

Some earlier content is stored behind [anyray: … retrieve ctx_…] markers.
Call anyray_retrieve with the ctx_ handle to read it back; if you remember
content that is no longer visible, find its handle with anyray_recall.

Keep it byte-identical across turns, because it sits in the cached prefix.

For a judgment task (a review agent, an auditor, a grader) make the line task-directive. An action agent that is missing bytes gets stuck and retrieves. A judge can assess the outline, produce something plausible, and never notice, so the failure mode is a shallow verdict that looks complete. Set the standard, not just the mechanism:

Never assess code you have not read in full. If a file you are commenting on
is behind an [anyray: … retrieve ctx_…] marker, call anyray_retrieve first.

Two mechanisms already reduce how often this fires. Files the live turn names, and files the agent is editing, are kept whole rather than outlined. Files the agent already evaluated are spared on later turns. The directive is the backstop for content the task reaches but did not name.

Reinforce it with an OpenClaw skill

OpenClaw loads markdown skills that teach tool use, so a minimal anyray-retrieval/SKILL.md with the same two sentences does the same job there. Not required, because OpenClaw models see the same in-band markers.