Strategy
A strategy is the unit of optimization. It inspects one request, transforms it or leaves it
untouched, and emits a decision recording what it did (kind, short summary, token/cost
estimates). It applies only when its own condition matches, and one that throws is skipped, so a
broken strategy can never break inference.
Each strategy is registered in the optimizer's REGISTRY
(optimizer/src/strategies/index.ts); a new optimization is a new strategy plus a registry entry.
The Library
One line each. Mechanics, parameters, and before/after examples follow in the reference below.
| Strategy | What it does | Default |
|---|---|---|
prompt_compression | Shortens long prompts and system messages. | on |
context_dedupe | Collapses repeated identical or near-identical tool outputs to the first copy plus changed lines. | on |
observation_mask | Retires stale tool observations behind a retrievable marker. | on |
context_compression | Shrinks tool outputs, logs, and RAG chunks. | on |
output_externalize | Moves a bulky tool output off the request, retrievable on demand. | off |
columnar_fold | Sends a record list's field names once instead of once per record. | off |
repeat_factor | Points a repeated block at its first copy, which stays inline. | off |
command_digest | Digests recognized dev-command output (test runs, grep, logs). | on |
code_graph | Keeps relevant symbols across multi-file reads. | on |
relevance_filter | Keeps only the relevant lines of a tool output. | on |
tool_pruning | Drops tools unlikely to be called. | off |
tool_schema_compression | Shrinks tool definitions while keeping all tools. | on |
client_prefix_compression | Swaps a known client's own boilerplate for curated short forms. | on |
semantic_cache | Serves a cached response for duplicate requests. | on |
param_tuning | Caps an over-large max_tokens. | off |
vision_ocr | Replaces text-only images with OCR'd text. | off |
provider_context_trim | Asks Anthropic to clear aged tool results provider-side, before billing. | on |
reasoning_budget | Downshifts reasoning effort on routine tool-resume turns. | off |
output_shaping | Asks for concise replies on routine tool-resume turns. | off |
thinking_trim | Removes re-sent copies of past reasoning. The current turn's thinking is untouched. | on |
window_budget | Crops low-relevance middle turns to fit the model's context window. | off |
cache_optimizer | Stabilizes the request's start so providers reuse their cache. | on |
Provider prompt caches bill only the new part of a request at full price; everything before
the cache_control breakpoints is a cheap cache read. Anyray never edits anything before a
breakpoint, so it can't invalidate the cache, and optimizations that would change the cached
prefix are skipped.
Which pipeline runs, and in what order, is per-deployment configuration: Optimizer pipeline. What happens when a strategy would be unsafe to apply is Guardrails.
Reference
What each strategy edits, when it runs, and which parameters it takes. Observe-only kinds, meaning meters and audits, are not listed.
observation_mask
Replaces an old, large tool result with a one-line marker, so it stops being billed every turn. The original stays retrievable, and failures, the current turn's input, and files the agent is editing are always kept.
It waits until the request fills the context window, and either bar starts it:
| Parameter | Default | Meaning |
|---|---|---|
minWindowFill | 0.35 | Share of the model context window the request must reach. |
minFillTokens | 70000 | Absolute request size that also starts masking. |
Both bars count the whole request, tool definitions included; set both to 0 to mask at any size.
A masked result keeps its marker on later turns, so the request stays byte-stable.
A client with no retrieval path gets a head-and-tail excerpt instead, under 5% of the result and
capped further by noRetrieveKeepChars (default 800). A client with a
real read path gets the full original.
On an api-key lane with no retrieval path it stands aside, because that lane bills every byte and
the marker would replace an output the agent is checking its fix against.
The two retrieval tools a client registers reach the model only from the first marker of a session on. Before that, the request on the wire is the base-URL lane's: the tools' presence alone changed how the agent worked a lint task, from a one-shot fix to thirty hand edits, before any output had been shortened. A session that never shortens anything never sees them.
turn 2 [tool] Read src/checkout.ts → 380 lines (≈ 9 KB)
turn 5 [tool] npm test → 2 failed, 126 passed (≈ 6 KB)
…
turn 12 [tool] Read src/tax.ts ← fresh input
turn 2 [tool] [anyray: observation masked · 9214 chars · retrieve ctx_9f… via anyray_retrieve]
turn 5 [tool] npm test → 2 failed, 126 passed ← kept: error
…
turn 12 [tool] Read src/tax.ts ← kept verbatim
command_digest
Recognizes known dev-command output and keeps what matters for that command. A test run keeps its
failures and its count line. grep output is bucketed by file, and repetitive logs collapse to
[×N]. The full output stays retrievable.
Grep hits that differ only in a timestamp, duration, or client IP fold into one line with a count.
Hits that differ by path, identifier, or status code stay separate, so 500 and 200 never merge.
It touches history only, so the current turn's results always pass through. On a cached session, a settled digest replays to identical bytes.
============ test session starts ============
platform linux -- Python 3.11.8, pytest-8.1.1 · collected 128 items
tests/test_auth.py ......F............... [ 22%]
tests/test_billing.py .................... [ 71%]
tests/test_api.py ..........F......... [100%]
================== FAILURES ==================
FAIL: test_login_lockout - assert False is True (tests/test_auth.py:42)
FAIL: test_refund_rounding - assert 999 == 1000 (tests/test_billing.py:88)
====== 2 failed, 126 passed in 3.41s ======
FAIL test_login_lockout tests/test_auth.py:42 assert False is True
FAIL test_refund_rounding tests/test_billing.py:88 assert 999 == 1000
2 failed, 126 passed in 3.41s
… digest · full output retrievable (retrieve ctx_a4f1)
code_graph
When an agent re-reads several source files in one request, code_graph keeps the functions in play
and the ones they call. It replaces the other bodies with a short marker and leaves the signatures
in place. Hidden bodies stay retrievable.
It builds a reference graph across the files and matches it to the current turn, not to how recent each file is. It judges what is central as a share of the code in front of it, so a larger read does not keep proportionally more, and with no clear target it outlines the whole file.
A file the agent is editing, or that the live turn names by path, is never outlined, so the model
composes its edits against real bytes. Inside the provider-cached prefix it outlines a settled file
only when that saves at least mintPaybackRatio (default 0.25) of the bytes the edit re-writes,
because a small outline never earns back a prefix rewrite.
The analysis is deterministic: no LLM, no embeddings, and it reads both brace and indentation languages.
# 6 files re-read this session · ≈ 48,000 chars
checkout.ts (asked about) charge(), refund()
pricing.ts (asked about) price()
tax.ts (imported) calcTax() ← called by price()
audit.ts (imported) log(), flush()
loyalty.ts (imported) award()
email.ts (imported) send()
# working set kept in full · ≈ 11,000 chars
checkout.ts charge(), refund() full body
pricing.ts price() full body
tax.ts calcTax() ← neighbor of price() full body
audit.ts log() → "[8 lines · retrieve ctx_7c]"
loyalty.ts, email.ts → outlined, bodies elided
relevance_filter
Builds a query from the latest message, ranks every line of the tool output, and keeps the top scorers plus a few leading lines for orientation. The user's own words are never ranked or trimmed. Removed lines stay retrievable.
It touches history only, and on cached traffic it ranks just the uncached tail. It stands aside for
a source file that code_graph will outline. Ranking that file line by line would
break the structure code_graph reads.
Hybrid mode adds a local embedding re-rank that catches paraphrases and synonyms. It runs in-network and falls back to lexical ranking.
find the 500 errors on /checkout
10.0.2.14 GET /health 200 2ms
10.0.2.14 GET /assets/app.js 200 5ms
10.0.9.3 POST /checkout 500 812ms
10.0.2.14 GET /home 200 8ms
10.0.9.7 POST /checkout 500 903ms
… 1,195 more lines … (≈ 52,000 chars)
find the 500 errors on /checkout ← kept verbatim
… [312 line(s) omitted by anyray · retrieve ctx_5a]
10.0.9.3 POST /checkout 500 812ms
10.0.9.7 POST /checkout 500 903ms
10.0.9.7 POST /checkout 500 774ms
… [180 line(s) omitted by anyray · retrieve ctx_5a]
(≈ 3,800 chars)
semantic_cache
When a request exactly repeats one already seen, Anyray serves the stored response and skips the provider.
A hit needs an exact match on the caller partition and on every field that can change the answer:
model, system and messages, tools, and all sampling params. Only stream with its delivery options
and a top-level prompt-cache directive are exempt, and request headers never count. Any other body
field is part of the key, so an unfamiliar control costs a miss instead of a wrong answer.
Entries are partitioned per caller: the tenant plus the attributed user, else the team, else the gateway's session id. A request that resolves none of those is neither served nor stored, so no caller ever sees another caller's response.
On a miss it also computes a shadow key that normalizes timestamps, UUIDs, and temp paths, then
records whether that key would have hit. That tier is measurement only, and its counters read from
GET /admin/optimizer/settings.
request #1 "What is our refund policy?" (gpt-4o) → provider call (~2.3 s)
request #2 "What is our refund policy?" (gpt-4o) → provider call (~2.3 s)
• billed twice
request #1 "What is our refund policy?" (gpt-4o) → MISS → provider call (~2.3 s)
request #2 "What is our refund policy?" (gpt-4o) → HIT → served from cache
• provider skipped → 0 tokens billed
• ~0 ms vs ~2.3 s upstream
context_dedupe
Collapses a tool output the agent has already seen. A re-read file or a re-run test suite becomes a one-line marker. A near-repeat keeps the changed lines. The first copy stays verbatim, and every collapsed copy stays retrievable.
It rules on a copy once, when that copy arrives, then replays the same verdict on later turns. The
request stays byte-stable for the provider cache. Content the agent pulled back with
anyray_retrieve is never collapsed again.
turn 3 [tool] Read src/billing.ts → 412 lines (≈ 11 KB)
…
turn 19 [tool] Read src/billing.ts → the same 412 lines, barely changed (≈ 11 KB)
turn 3 [tool] Read src/billing.ts → 412 lines (≈ 11 KB)
…
turn 19 [tool] [anyray: near-duplicate delta-collapsed — 98% identical to the
412-line output above; 3 changed line(s) below · retrieve ctx_8k2p…]
@@ -41,1 +41,1 @@ -const RATE = 0.07; +const RATE = 0.075;
@@ -388,0 +389,1 @@ +export const auditRate = () => RATE;
context_compression
The catch-all for bulky tool output: it minifies JSON, trims over-long string fields, and trims
common shapes such as diffs, grep, lint reports, stack traces, and repetitive logs. What it
removes stays retrievable.
Long arrays are kept whole (maxArrayItems: 0). A cut list still reads as a complete list to the
model, so a value outside the kept window becomes a confident wrong answer. Set maxArrayItems only
for data that is safe to cut from the end.
It runs after the specialists, so it cannot mangle their input, and the current turn's output is
never touched. On a client that cannot call POST /v1/retrieve it still compresses history, but it
stashes nothing and emits no handles.
[
{ "id": 1, "name": "alice", "email": "a@ex.com", "active": true },
{ "id": 2, "name": "bob", "email": "b@ex.com", "active": true },
{ "id": 3, "name": "carol", "email": "c@ex.com", "active": false },
… 797 more rows …
] (≈ 41 KB)
[{"id":1,"name":"alice","email":"a@ex.com","active":true},
{"id":2,"name":"bob","email":"b@ex.com","active":true},
{"id":3,"name":"carol","email":"c@ex.com","active":false},…] (≈ 17 KB)
Every row survives, and the saving comes from whitespace and repeated structure.
output_externalize
Replaces a whole bulky message with a compact handle and stashes the original. The agent pulls the
exact bytes back with POST /v1/retrieve. It applies to tool outputs at or above minChars
(default 4000), keeps the copy for ttlSeconds (default 86400), and skips the current turn.
It stands down in two cases. It runs only when the durable tier can persist and the client can call
POST /v1/retrieve. It is also registered as cache-busting, so it is suppressed on prompt-cached
traffic.
turn 4 [tool] psql → 1,842 rows (≈ 180 KB)
turn 4 [tool] [anyray:externalized 184320 chars · retrieve ctx_9f3a]
columnar_fold
A tool that returns a list of records repeats the same field names on every record.
columnar_fold rewrites the list as one column header plus one row per record, and lifts out any
field with the same value on every row.
Nothing is dropped, shortened, or stashed. The folded payload is unfolded and compared against the original first, and the tool result passes through untouched if anything differs. Key order inside a record is not preserved, and JSON does not treat key order as meaningful.
| Parameter | Default | Meaning |
|---|---|---|
| roles | tool, function | Message roles it applies to. |
minChars | 400 | Minimum payload size. |
minRecords | 4 | Minimum records in the array. |
minSavedChars | 120 | Skip a fold that would not save at least this much. |
Records with different key sets are left alone. A JSON body after a prose question is folded and the prose is kept, but a payload with prose after the JSON is declined. The same tool result folds to the same bytes every turn, so a cached prefix holding it stays byte-identical.
[{"object":"order","id":"ord_1N00","currency":"usd","amount":1995,"status":"paid"},
{"object":"order","id":"ord_1N01","currency":"usd","amount":4200,"status":"failed"},
… 498 more records …]
{"__anyray_columns__":["id","currency","amount","status"],
"__anyray_constant__":{"object":"order"},
"__anyray_rows__":[["ord_1N00","usd",1995,"paid"],
["ord_1N01","usd",4200,"failed"],
… 498 more rows …]}
repeat_factor
Long blocks of text repeat across tool results that are not duplicates of each other: the same
licence header on nine file reads, the same schema echoed by three query results. repeat_factor
finds the longest repeated blocks in the request, keeps the first copy as it is, and replaces each
later copy with one line.
Nothing is stashed. The copy a marker points at is still in the request, so the model resolves the reference by reading, not by calling a tool. The first copy is never rewritten, and neither is a span another strategy already claimed.
| Parameter | Default | Meaning |
|---|---|---|
| roles | tool, function | Message roles it applies to. |
minLines | 8 | Minimum lines in a repeated block. |
minRunChars | 400 | Minimum characters in a repeated block. |
maxLinesScanned | 20000 | Scan cap per request, counted from the first message. |
It removes only redundancy, so its output is a function of bytes that are already settled. A cached prefix holding a marked block stays byte-identical.
[tool] read src/billing.ts 62-line licence header + imports, then the code
[tool] read src/checkout.ts the same 62 lines, then different code
[tool] read src/pricing.ts the same 62 lines, then different code
[tool] read src/billing.ts the 62 lines, kept verbatim
[tool] read src/checkout.ts [anyray: 62 identical lines elided — repeat of the
62-line block starting "// Copyright (c) 2026 …"
earlier in this conversation]
[tool] read src/pricing.ts the same one-line marker
tool_pruning
Drops tools the model is unlikely to call, which trims schema overhead on every request, and it errs
toward keeping. A tool survives if its name or description shares a topic word with the
conversation. Generic words like "get" or "find" do not count, and neither does wording most
descriptions share. MCP tools (mcp__…) are never pruned.
Deciding again every turn would shift the tools prefix and lose the cache. On a pinned session it
decides once and replays the same set byte-for-byte. It decides again when the tool list changes,
when tool_choice forces a pruned tool, or after a tool-miss. Without pins it stands aside on warm
cached traffic.
task: "comment on the P0 bugs in the PAY project"
tools (3): (≈ 2,400 schema tokens)
jira_search_issues "Search issues in a Jira project by JQL."
slack_post_message "Post a message to a Slack channel."
datadog_query_metric "Query a Datadog metric over a range."
tools (1): (≈ 800 schema tokens)
jira_search_issues "Search issues in a Jira project by JQL."
prompt_compression
Collapses repeated whitespace and drops duplicate sentences and paragraphs in system and user
messages over 400 characters. It keeps the first copy and preserves line and list structure. Code
passes through byte-for-byte, and tool result blocks are never touched.
Each message is judged on its own bytes. One is rewritten only when the pass saves at least
minSavedChars (24) and minSavedRatio (3%) of it, because the system prompt and the first user
turn sit in the cached prefix, where a trivial rewrite is re-billed in full on a bypassed turn. On
an api-key lane with no retrieval path it stands aside, since a user turn there can carry a pasted
tool report.
You are a senior support engineer.
Read the ticket below and reply. Be concise. Be concise. Be concise.
Then suggest next steps. Then suggest next steps.
You are a senior support engineer.
Read the ticket below and reply. Be concise.
Then suggest next steps.
tool_schema_compression
Shrinks each tool's free-text description: it collapses whitespace, strips boilerplate like "This tool allows you to…", and rewrites filler. Types, enums, and required fields stay byte-for-byte intact, so every tool stays callable. It also shrinks parameter descriptions, where most of a verbose schema's prose sits. Details that tell the model what to pass are kept, such as "for example PAY or CORE".
It pairs with tool_pruning: pruning drops irrelevant tools, and this shrinks the
ones that stay. It reads only each tool's own bytes, so it is deterministic and cache-safe.
On ten verbose MCP schemas the default pass trims about 8% of the request with no loss. The optional
maxDescriptionChars cap takes that to about 32%, cutting at a sentence or a word boundary.
search_kb: "This tool allows you to search the knowledge base. Use it
whenever you need a citation." (~110 chars)
create_ticket: "This tool allows you to create a support ticket. Please be
sure to fill every field." (~102 chars)
search_kb: "search the knowledge base. Use it whenever you need
a citation." (~76 chars)
create_ticket: "create a support ticket. Fill every field." (~44 chars)
client_prefix_compression
Some clients ship a large fixed prefix on every request. Claude Code's system prompt and built-in tool descriptions run to about 12k tokens, re-read on every call of a session. A generic compressor gets 2% of that, so this strategy carries a curated table instead: for each known passage, the exact bytes a shipped version sends and a hand-written short form that keeps every instruction.
A match is byte-for-byte, so the same original always maps to the same replacement, and anything that does not match is forwarded untouched. That includes another client version, a model-specific variant, your own CLAUDE.md, and the per-host environment lines. The table covers Claude Code 2.1.257 and 2.1.260 on Sonnet 5, and a release that changes a passage stops matching it until an entry is added.
It runs before tool_schema_compression and
prompt_compression, which then see the curated text. The gateway remembers
each rewrite and replays it on a bypassed turn, so a timeout or a cooloff forwards the same prefix
the warm turns did.
| Parameter | Default | Meaning |
|---|---|---|
tables | ["claude_code"] | Curated tables to apply. claude_code_tools and claude_code_system each select one half. |
Bash tool description: 9,593 chars
Static system prompt passages: 24,570 chars
Bash tool description: 6,727 chars
Static system prompt passages: 15,065 chars
param_tuning
Caps an over-large max_tokens. The default cap is 65536. This is a guardrail against runaway
completions, not a spend saver. You are billed for the tokens actually generated, so a lower ceiling
does not lower the bill. A request already under the cap passes through unchanged.
max_tokens: 100000
max_tokens: 65536
vision_ocr
Swaps a text-only image for its text, which turns costly vision tokens into cheap text tokens. A screenshot, stack trace, or log qualifies. Only high-confidence text images are swapped, and diagrams, charts, and photos pass through untouched. OCR runs locally with no extra model call, and the original stays retrievable.
[user] <image: screenshot of a 30-line Python stack trace>
(≈ 1,000 vision tokens)
[user] [anyray: text extracted from a pasted image
(240 words, 94% confidence) · retrieve ctx_3a]
Traceback (most recent call last):
File "app.py", line 42, in charge …
(≈ 320 text tokens)
provider_context_trim
On a Claude request over a size threshold, in any billing mode, it adds Anthropic's native
context-management edit (clear_tool_uses_20250919) and its beta header. Anthropic then clears aged
tool results before billing. API-billed requests stop paying for those tokens, and subscription
seats stop burning usage limit on them.
The threshold scales with the model's context window: 30% of the window, never below 60,000 tokens,
so a long-context lane is not trimmed while mostly empty. An explicit triggerTokens setting always
wins.
It only labels the request, and never reads, rewrites, or removes message content. A request that
carries its own context_management block passes through verbatim.
It shares the trimming job with observation_mask, which takes precedence, and
Anyray reports no savings for a provider-side clear.
POST /v1/messages claude-* (metered API key)
long agent trajectory ≈ 72,000 input tokens
aged tool results are re-billed every turn
same request body + injected context_management:
edits: [{ type: clear_tool_uses_20250919, trigger: 60,000 input tokens,
keep: 3 tool uses, clear_at_least: 5,000 tokens }]
→ Anthropic clears aged tool results before billing
reasoning_budget
On a routine tool-resume turn, where the fresh input is tool results and nothing failed, it lowers reasoning effort for metered reasoning models. A client-set value only ever moves down, across three lanes:
- Anthropic thinking budget. Caps a client-set
thinking.budget_tokens, and never adds or removesthinking. - Anthropic effort. Sets
output_config.efforton allow-listed Claude models when the client sent none, and clamps a higher client value down. - OpenAI output controls. Sets GPT-5
verbosity, orreasoning_efforton allow-listed models, and clamps higher client values down the same way.
On a pinned session the first decision mints a pin, so later turns replay the same value and the session pays at most one cache invalidation. Values back off on a turn whose tool result carries an error, and a model switch drops the pin.
Enable it as a canary first: it claims zero estimated savings, and the measure is the output-token and session-length delta.
[assistant] run tests with high reasoning
[tool] 128 passed (fresh input)
thinking.budget_tokens: 31999
[assistant] run tests with high reasoning
[tool] 128 passed (fresh input)
thinking.budget_tokens: 8192
output_shaping
The one output-side strategy. On a routine tool-resume turn it appends one short advisory to the end of the request. The advisory asks the model to reference file paths and diffs instead of restating context. It edits nothing already in the request, so cached prefixes stay byte-identical.
| Parameter | Default | Meaning |
|---|---|---|
minAssistantTurns | 2 | Assistant turns required before a tool-resume counts as routine. |
skipOnErrorTurn | true | Skip fresh tool results the provider flagged with is_error. |
It reports zero estimated savings. The advisory steers the model rather than capping it, so only the measured output-token delta is honest. The advisory is visible text in the request, and some coding agents flag in-content instructions as prompt injection. Try it on one lane first.
[assistant] called a tool
[tool] routine tool result (fresh input)
[assistant] called a tool
[tool] routine tool result (fresh input)
+ [anyray:output-guidance] Be concise: reference file paths and diffs
instead of restating context or re-printing unchanged files.
thinking_trim
Shown in the console as Thinking replay trim, it removes replay, not the model's reasoning on
the current turn (that is reasoning_budget). Agentic clients resend finished
thinking blocks with every request, and that replay is billed input every time.
It keeps the opening plan of each tool loop (keepLoopHead, default 1), then trims a block once
everything that block names (paths, symbols, error IDs) also appears in the same assistant message.
Blocks with no readable text are still billed on replay, so cutOpaqueChain (default true) drops
them past the loop head.
Three kinds of reasoning are always kept: encrypted redacted_thinking, reasoning that recovers
from a tool error (skipOnErrorTurn), and reasoning on a turn that called an Edit or Write tool
(protectMutationToolCalls, default true). Each carries state a tool receipt does not, because a
mutation record says what changed, never why. Setting keepLoopHead to 0 is an explicit drop-all
and overrides all three.
Past those keeps the first cut is sticky, so old verdicts stay byte-stable, and the classifier
parameters (redundancyThreshold, externalizationThreshold, maxScanChars) default to auto.
[assistant] (thinking: plan, 1.2k chars) (tool_use run_tests)
[tool] 128 passed
[assistant] (thinking: step, 800 chars) (tool_use read_file)
[tool] contents…
[assistant] (thinking: plan, 1.2k chars) (tool_use run_tests)
[tool] 128 passed
[assistant] (tool_use read_file)
[tool] contents…
window_budget
When a session's history outgrows the model's context window, window_budget evicts middle messages
and replaces each with a retrievable placeholder. It scores messages by relevance against the live
turn plus recency. Task setup (keepLeading), the live turn (keepRecent), and pinned roles are
never evicted. Every evicted original comes back through POST /v1/retrieve.
The budget is dynamic: it fits the model's real context window as the gateway learns it, with a 0.95
safety margin. When no window is known, nothing is cropped, and maxTokens sets a fixed operator
ceiling instead. maxTokens: 0 disables cropping.
It is the one strategy that drops whole messages, and settled crops replay byte-identically, so they never re-bust the provider cache.
[user] task setup (kept: leading)
[tool] 12k-token exploration dump (low relevance → evicted)
[tool] shorter, on-topic result (kept: relevant)
[user] the live question (kept: recent)
[user] task setup
[tool] [anyray: 12k tokens cropped · retrieve ctx_…]
[tool] shorter, on-topic result
[user] the live question
cache_optimizer
Runs last and keeps the provider's cache discount alive. It puts the tools block in a
byte-identical order every turn. On Anthropic it inserts cache breakpoints for a cached-read
discount. On selected OpenAI models it stamps a deterministic prompt_cache_key when the client set
none.
It is lossless. It only reorders tools and adds cache hints, and it never touches prompt content.
It backs off when the client already manages caching.
turn 1 tools: [search, book_hotel, get_weather]
turn 2 tools: [get_weather, search, book_hotel] ← order drifts
turn 3 tools: [book_hotel, search, get_weather]
→ prefix changes → provider cache misses → full price
every turn tools: [book_hotel, get_weather, search] (fixed)
+ cache_control on the last tool, system, and settled history
→ prefix identical → provider cache hits → cached-read rate