Skip to main content

Agent tool-loop reference

Close the retrieval loop in an agent SDK​

Register both lanes in the same agent setup: a source hook runs before a tool result enters the transcript, and the MCP server supplies the read-back path. MCP registration alone does not install a source hook. For the Claude Agent SDK, use the complete options below. For another tool loop, call the same PostToolUse(output, tool) function before appending each result. Send x-anyray-metadata: {"tool":"sdk-agent"} on its inference requests so the console can identify this manually integrated lane. Plain SDK calls without a tool loop keep their existing client ID.

When the optimizer elides bulk it leaves a ctx_… handle in the prompt, and the model needs a way to read the original back. The gateway keeps retrieval-dependent strategies disabled until it receives two things:

FactWhat proves it
A callable retrieval pathA declared anyray_retrieve tool, or a registered MCP server.
Fresh authenticated livenessAuthenticated MCP, a heartbeat, or model retrieval activity.

Nothing else opts a client in. Not a base URL, an API key, or an SDK user-agent. Starting a server beside a client that cannot expose MCP tools to the model does not close the loop, and neither does implementing POST /connect/retrieve without declaring a tool the model can call. There is no retrieval-capability environment variable and no manual flag.

The simplest opt-in is that the gateway is itself a remote MCP server. POST /mcp (streamable HTTP) serves anyray_retrieve and anyray_recall, gated by the same client key the agent already sends. Registering it, allowing the two tools, and checking that it latched are three steps on the SDK setup guide; OpenClaw's equivalent entry is in its guide. The rest of this page is the depth behind them.

A workflow of one-shot calls needs none of this. A summarize, extract, write pipeline, a batch job, or an embeddings-only service cannot call tools, so it runs the no-retrieve lane automatically and nothing is elided behind a handle the flow could not resolve.

The gateway MCP endpoint​

POST /mcp speaks the streamable-HTTP transport: one JSON-RPC message per POST, notifications acknowledged with 202, and a signed Mcp-Session-Id pinning the initial announcement. Instruction and reconnect behavior. Successful initialization and other valid authenticated messages renew the key's 6-hour retrieval lease. Rejected initialization does not claim retrieval capability.

The host that sent the handshake is the process exposing tools to the model, so retrieval-dependent optimization unlocks after initialization without waiting for a retrieval call. The endpoint advertises the retrieval pair only, because the stdio server's local-workspace tools (anyray_search, anyray_read) act on files a remote endpoint does not have.

Hosted connectors​

Zero client-side runtime: the platform calls your gateway's /mcp from the outside.

Anthropic's Messages API needs all three parts. mcp_servers, its paired mcp_toolset entry in tools (a server entry without one is a validation error), and the beta header:

Request params (header: anthropic-beta: mcp-client-2025-11-20)
"mcp_servers": [
  { "type": "url", "url": "https://<gateway>/mcp", "name": "anyray",
    "authorization_token": "<your-gateway-key>" }
],
"tools": [ { "type": "mcp_toolset", "mcp_server_name": "anyray" } ]

Or the OpenAI Responses hosted tool, inside tools:

{ "type": "mcp", "server_label": "anyray", "server_url": "https://<gateway>/mcp",
  "headers": { "x-anyray-api-key": "<your-gateway-key>" }, "require_approval": "never" }
ConstraintDetail
Same public URLUse the gateway URL the SDK already sends requests to. The gateway credits a connector as the read path only when it targets its own /mcp.
Publicly reachableThe platform calls from the outside, so a VPC-private gateway cannot use hosted connectors. Register the MCP server in the agent runtime instead.
First-party APIs onlyPer the providers' own compatibility notes, Anthropic's connector is not available on Amazon Bedrock or Google Cloud.
OpenAI: Responses API onlyOn chat.completions, declare and implement the tool yourself, or register the MCP server in your agent runtime.

A deployment routing Claude through Bedrock or Vertex loses nothing else. The declared-tool path and the runtime MCP entry are provider-independent: the marker rides the prompt, the tool rides the request, and the agent's own loop calls the gateway directly. The declaration makes the tool selectable, and authenticated MCP activity supplies the live lease.

Plain HTTP tools (your own tool loop, no MCP)​

For an agent that runs its own tool loop with no MCP support. Three independent things must be true:

1
The declaration
Proves the model can select the tool on this turn. It matches on the tool name (anyray_retrieve, or an mcp__…__anyray_retrieve-style namespaced form), so the name is load-bearing and must not be aliased.
2
The implementation
Against POST /connect/retrieve. This is what makes the handle resolvable once the model calls it.
3
The liveness signal
Proves the implementation is running. Send an authenticated empty POST /connect/mcp-heartbeat when the worker starts, and every 5 minutes while it can execute tool calls. Stop heartbeats before shutting the executor down. The lease expires after 6 hours without verified activity.

A declaration qualifies for that turn. The lease covers later turns whose tool catalog is deferred.

The declaration shape follows whichever dialect you already call. Both satisfy the gateway:

Anthropic dialect
ANYRAY_RETRIEVE_TOOL = {
    "name": "anyray_retrieve",
    "description": (
        "Fetch the full original text behind an Anyray handle. When a message "
        "contains a marker like [anyray: ... retrieve ctx_abc123], pass that "
        "ctx_ handle here to read what it replaced."
    ),
    "input_schema": {
        "type": "object",
        "properties": {"handle": {"type": "string"}},
        "required": ["handle"],
    },
}

The description is doing real work. The gateway unlocks retrieval on the name, but the model only calls the tool if it knows what a ctx_… marker is. Then implement it, in either dialect:

Implement
def anyray_retrieve(handle: str) -> str:
    r = requests.post(
        "http://<gateway>:8787/connect/retrieve",
        headers={"x-anyray-api-key": "<your-gateway-key>"},
        json={"handle": handle},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()["content"]

The response is {"status":"ok","content":"…","handle":"ctx_…"}. startLine, endLine, and grep are optional body fields for reading back a slice, and an unknown or expired handle answers 404. The stdio server below wraps this same route, so neither path is more capable than the other.

Stdio MCP (Connect already installed)​

Where anyray-connect is installed, its stdio server is equivalent to the remote endpoint, being a wrapper over the same routes, and also serves the local-workspace tools anyray_search and anyray_read. For the Claude Agent SDK's TypeScript query options, the complete server entry is paired with its source hook:

Claude Agent SDK options
settingSources: ['user', 'project', 'local'],
mcpServers: { anyray: { type: 'stdio', command: 'anyray-connect', args: ['__anyray-mcp-server', 'claude'] } },
settings: { hooks: { PostToolUse: [{ matcher: '*', hooks: [{
  type: 'command', command: 'anyray-connect __anyray-hook --managed-source', timeout: 3
}] }] } }

The hook trims before the result enters the transcript. It requires successful retrieval in the current main transcript and checks that the returned handle reads back the whole original. Missing proof, a failed request, or a two-second deadline leaves the original untouched. Restricted subagents stand aside. Managed hosts without a binary receive a native HTTP hook; its event has no callable-tool evidence, so the gateway currently returns a no-op. No key-level latch authorizes that lane.

doctor --json and status --json report sourceHook.installation and sourceHook.retrieval separately. Retrieval reports expire after one minute and never authorize a trim.

The Agents row polls its key record. Active · observing source trim is the grace state; a successful trim changes it to Active once retrieval is connected. Active · source trim not seen means at least seven days between the start of observation (or last trim) and the latest eligible inference request, with at least 100 successful inference requests since then. Idle time alone cannot trigger it. This diagnoses absent evidence, not a promise that a workload must trim. Only Claude Code has an automatically installed source hook today. sdk-agent identifies an explicitly integrated harness; other client tools report source-hook support as unavailable. Older gateways show Needs gateway upgrade instead of diagnosing the host.

This mounts the exact stdio command anyray-connect __anyray-mcp-server claude. The executable must be a durable Connect install on the agent process's PATH, with an enrolled profile using the same key and gateway as the model requests. On MCP initialization the server exposes anyray_retrieve and anyray_recall. Connect renews that evidence every 5 minutes while the initialized stdio transport is alive, and the gateway expires it 6 hours after the last accepted heartbeat or model retrieval call.

Custom SDK callback​

For an explicitly integrated sdk-agent harness, use this callback with its MCP read path.

Source hook and read-back together
import type { Options } from "@anthropic-ai/claude-agent-sdk";

const gateway = "https://gateway.example"; // deployment's public gateway URL
const headers = { "x-anyray-api-key": process.env.ANYRAY_CLIENT_KEY! };

async function PostToolUse(output: unknown, tool: string): Promise<unknown> {
  if (typeof output !== "string" || !output) return output;
  try {
    const response = await fetch(gateway + "/connect/optimize-output", {
      method: "POST",
      headers: { ...headers, "content-type": "application/json" },
      body: JSON.stringify({ output, tool }),
      signal: AbortSignal.timeout(5000),
      redirect: "error",
    });
    if (!response.ok) return output;
    const result = await response.json();
    return result.optimized === true && typeof result.output === "string"
      ? result.output : output;
  } catch { return output; }
}

const anyrayOptions: Options = {
  mcpServers: {
    anyray: { type: "stdio", command: "anyray-connect",
      args: ["__anyray-mcp-server", "profile"] },
  },
  hooks: {
    PostToolUse: [{ hooks: [async (input) => {
      if (input.hook_event_name !== "PostToolUse") return {};
      const output = await PostToolUse(input.tool_response, input.tool_name);
      return output === input.tool_response ? {} : {
        hookSpecificOutput: { hookEventName: "PostToolUse", updatedToolOutput: output },
      };
    }] }],
  },
};

Keep the key in the host's existing secret environment. The authenticated POST /connect/optimize-output contract accepts {"output": "…", "tool": "…"} and returns {"optimized": true, "output": "…"} when it trims. No-op responses set optimized: false; empty inputs and diagnostic probes may omit output. Preserve the original on a no-op, timeout, HTTP error, or invalid replacement. This minimal hook uses the existing terminal-trim lane; it does not request reversible trimming. MCP supplies read-back for gateway elisions. Remove the hooks entry to stop source trimming.

How the model knows when to call it​

You do not need to teach the happy path. Every elision leaves a marker in-band, at the spot the content was, and the marker names the verb:

[anyray: observation masked · 330509 chars · retrieve ctx_9f3a2b via anyray_retrieve]
Do not rewrite the marker or the tool description

This wording is load-bearing and measured. Before the marker named the tool, the fleet resolved 0.4% of emitted handles. Naming anyray_retrieve outright is what made models call it.

Reinforce it with a system-prompt line

This is the only teaching channel that reaches every runtime, and the main way to surface anyray_recall, which nothing in-band points at once a marker has scrolled out of view or been compacted away.

Some earlier content is stored behind [anyray: … retrieve ctx_…] markers.
Call anyray_retrieve with the ctx_ handle to read it back; if you remember
content that is no longer visible, find its handle with anyray_recall.

Keep it byte-identical across turns, because it sits in the cached prefix.

For a judgment task (a review agent, an auditor, a grader) make the line task-directive. An action agent that is missing bytes gets stuck and retrieves. A judge can assess the outline, produce something plausible, and never notice, so the failure mode is a shallow verdict that looks complete. Set the standard, not just the mechanism:

Never assess code you have not read in full. If a file you are commenting on
is behind an [anyray: … retrieve ctx_…] marker, call anyray_retrieve first.

Two mechanisms already reduce how often this fires. Files the live turn names, and files the agent is editing, are kept whole rather than outlined. Files the agent already evaluated are spared on later turns. The directive is the backstop for content the task reaches but did not name.

Reinforce it with an OpenClaw skill

OpenClaw loads markdown skills that teach tool use, so a minimal anyray-retrieval/SKILL.md with the same two sentences does the same job there. Not required, because OpenClaw models see the same in-band markers.