Strategy reference

Defaults, parameters, and limits for each strategy. To see what each one changes, start at the Strategy library.

Managed Claude source hooks run before tool results enter the transcript. They stand aside unless the current lane can retrieve the original. Source-hook behavior.

The Library

On by default:

Strategy	What it does
observation_mask	Retires stale tool observations behind a retrievable marker.
code_graph	Keeps relevant symbols across multi-file reads.
cache_optimizer	Stabilizes the request's start so providers reuse their cache.
relevance_filter	Keeps only the relevant lines of a tool output.

columnar_fold and repeat_factor are on for every client. command_digest is on in its lossless lane only: it folds repeated blocks in fresh test output and changes nothing else.

content_census and cache_lint are also on. They only measure and never change a request. content_census also reports the tokens a trim would have saved on a large pasted log.

Off by default

Enable these on a deployment you have measured. Each one changes what the model sees, or needs a client capability.

Strategy	What it does
command_digest (full digest)	Digests recognized dev-command output (test runs, grep, logs).
thinking_trim	Removes re-sent copies of past reasoning. The current turn's thinking is untouched.
context_dedupe	Collapses repeated identical or near-identical tool outputs to the first copy plus changed lines.
context_compression	Shrinks tool outputs, logs, and RAG chunks.
repeat_factor	Points a repeated block at its first copy, which stays inline.
prior_output_repeats	Points a Codex output's repeated lines at the earlier output in context. Codex seat lane only, off by default.
prompt_compression	Shortens long prompts and system messages.
tool_schema_compression	Shrinks tool definitions while keeping all tools.
provider_context_trim	Asks Anthropic to clear aged tool results provider-side, before billing.
output_externalize	Moves a bulky tool output off the request, retrievable on demand.
tool_pruning	Drops tools unlikely to be called.
client_prefix_compression	Swaps a known client's own boilerplate for curated short forms.
param_tuning	Caps an over-large max_tokens.
reasoning_budget	Lowers reasoning effort: Claude effort once per session, the rest on routine tool-resume turns.
output_shaping	Asks for concise replies on routine tool-resume turns.
window_budget	Crops low-relevance middle turns to fit the model's context window.
semantic_cache	Serves a cached response for duplicate requests.

Turn one on for everyone in the optimizer settings, for a team or model with a per-target rule, or for a single request with the x-anyray-optimization-kinds header. A rule's disable always wins.

Cache-safe by default

Provider prompt caches bill only the new part of a request at full price; everything before the cache_control breakpoints is a cheap cache read. Cache guards suppress unstable prefix edits. Replay-capable strategies may make a bounded, pinned rewrite and replay it on later turns, refreshing any retrieval handle. The session gate measures whether that cost pays back on the tenant's traffic. See it step by step.

API-key lanes

When enabled, prompt_compression and context_dedupe also run on API-key (billed) lanes. On a real repository, context_dedupe sometimes rewrote 19k to 29k cached tokens to save about 2k read tokens a turn, which never paid back inside a session. Watch cache writes and session cost on your API-key traffic, and turn the strategy off for API-key lanes where they get worse. Edits it already made in a session keep replaying. Subscription seats always run both strategies.

Set it on the strategy, or scope it with a per-target rule:

{
  "strategies": [
    { "kind": "context_dedupe", "enabled": true, "params": { "runOnBilledLane": false } }
  ],
  "overrides": { "rules": [
    { "when": { "metadata": { "experiment": ["billed-dedupe"] } },
      "params": { "context_dedupe": { "runOnBilledLane": true } } }
  ] }
}

Only the JSON value true runs the strategy on an API-key lane. The string "true" does not.

Pricing switches

Every edit that can cost a cache rewrite or a read-back is priced in dollars before it is made, on one token count, one horizon of later turns and one read-back chance per edit. An edit that does not pay is skipped and its strategy reports why (strategyDeclines). The config's economics block turns each part of that price on or off. unifiedRuler, enforceOpenLanes, and horizonSlopes default to false, the rest to true.

Switch	On	Off
tokenizer	Prices read real token counts of each message (the model's own tokenizer, or Claude's published vocabulary scaled to billed tokens)	Prices read the size estimate
unifiedRuler	Sizes are whole-message JSON at the density the gateway measured for the model	Each check keeps its older size estimate
realTurnHorizon	The context-ceiling cap on later turns grows per real turn	It grows per exchange, which can count one turn twice
measuredRetrieval	A tenant's measured read-back rate, for the same kind of content (source or output), raises the read-back chance	Built-in chances only
priceAllLanes	Edits after the cache breakpoint, repeat_factor's folds inside the cached prefix and thinking_trim's running-loop strips are priced	They run unpriced, as before
enforceOpenLanes	observation_mask under context pressure, or on a route with no prompt cache, drops masks that do not pay	The price is reported and every mask goes out
tailAdmission	An edit must also pay if the session ends early (the length three sessions in four reach)	The average session length decides alone; payback_tail is never reported
freshTailGroups	Prices fresh-tail edits together and admits retrieval-tool activation when the suffix pays for it	Prices edits separately once retrieval tools are active; holds suffix tool activation
quartileGrowth	The context-ceiling cap assumes later turns grow like a typical large turn (the upper quartile)	It assumes every later turn grows like the largest one so far
markedHeaderRead	When the request marks its system prompt for caching, an edit to an earlier message prices the tools and system prompt as read from cache	On a route without a measured cache window, the edit also prices the tools and system prompt as re-written
requestPlanning	Selects compatible edits across strategies and prices shared rewrite and retrieval setup costs once	Each strategy selects and prices its own edits
subsetSelection	Searches smaller groups when the initial edit group fails; each group must pass the full price and enabled tail checks	Only removes costly edits from a group that already passes
horizonSlopes	Caps main-thread gate and planner horizons, including explicit overrides, at the ANY-1211 per-tool slope	Uses the existing horizon; defaults off pending ANY-1157's paired bench

requestPlanning defaults to true. With route pricing evidence, the optimizer chooses edits across strategies against the final replayed request. The whole group must pay at the expected session length and the enabled early-ending bound. Required handles must be stored before any selected edit is published. Set economics.requestPlanning to false to restore per-strategy selection, including its freshTailGroups and subsetSelection controls. Explicit legacy payback ratios and pressure lanes with enforcement off keep their existing behavior.

observation_mask.params.mintSharedRetrieval defaults to false. With request planning on, setting it to JSON true prices retrieval once for selected masks that share a handle and restore identical content. Missing or conflicting identities keep separate charges. It also uses past cache-expiry observations to price waiting before a warm rewrite. Retry requests count once; changed history or pricing policy restarts the estimate. The estimate never grants cold pricing on a warm request.

Experimental pricing

Shared-handle pricing can trigger costly rewrites too early. Keep it off outside controlled experiments. Setting it back to false keeps request planning enabled.

With request planning off, freshTailGroups defaults to true. Set it to false to price each edit separately. A group is withheld if any required handle cannot be stored.

A rule's economics block replaces the fields it names for the requests it matches. Changing a switch restarts the per-session gate's evidence for the strategies it prices, keeping their current state.

Set subsetSelection to false to restore the previous selection algorithm.

Rule

{
  "when": { "metadata": { "experiment": ["mean-admission"] } },
  "economics": { "tailAdmission": false }
}

Reference

What each strategy edits and when it runs. Observe-only kinds, meaning meters and audits, are not listed.

observation_mask

Replaces an old, large tool result with a one-line marker, so you stop paying for it every turn. The agent can fetch the original back when it needs it.

It never masks a failed tool call, the current turn, a source file the agent read, a file the agent is editing, or the latest completed exchange. It masks freely as a turn nears its context limit (the client's own limit when it declares one, else the model's window), and below that only where the payback check admits it.

Masking a result that is already in the provider's prompt cache forces a cache rewrite. Anyray masks it only when the saving pays for that rewrite plus a possible read-back. Later turns count toward the payback, so short sessions are left alone.

On a measured Bedrock rewrite region, each message keeps its cache-write TTL. A 1-hour system cache does not raise a 5-minute message rewrite to the 1-hour rate. If a message spans both TTLs, the estimate uses the higher rate for its retained bytes.

Native Claude requests also account for the gateway's resolved cache lifetime when pricing the current write.

mintAnchor: true opts into pricing from a live earlier request tail, past the measured window on explicit Bedrock routes. It reads { index, hash } attestations from cacheAnchorTails, which the Anyray gateway does not send yet, so today the param has no effect. After every other strategy has run, the optimizer checks each hash against the request it is about to send and prices again only a trim the mask declined on its premium. A trim admitted there places one marker if the tail block is cacheable, the four-breakpoint rule has room and its TTL fits the surrounding markers. It defaults to false until a paired session bench passes. mintWholePrefixPrice: true disables it.

For an agent that re-sends tool output as an assistant message, such as LangGraph, add assistant to its roles. It is off by default because it also masks the model's own long replies on other agents. An assistant message at the very end of the request stays inline, and system or developer messages are never masked.

command_digest

On by default (lossless lane only)

Recognizes known dev-command output and keeps what matters for that command. The full output stays retrievable.

• A test run keeps its failures and the count line. Each failure keeps its assertion (expected, got, want, pytest's E lines), even under a long stack trace.
• grep output is grouped by file. Hits that differ only in a timestamp, duration, or client IP fold into one line with a count. Hits with a different path, id, or status code stay apart, so 500 and 200 never merge.
• Repeated log lines collapse to [×N].

Identical blocks inside one test output become references to the first copy, with no read-back needed. A test log the agent saved to a file and read back with line numbers folds the same way, and each reference names the line range it replaces. This is the only edit the shipped default makes (firstSightOnly: true), and only on output that arrives this turn and is provably not cached yet. Set firstSightOnly: false to add the lossy digests above, which edit older turns.

A digest of cached history has to pay for the cache rewrite it forces. Once a session sits idle past its prompt-cache lifetime, the provider bills the whole prefix again whatever is sent, so the rewrite costs nothing extra on that turn. The config's coldTurn.premiumKinds lists the strategies that price a mint without the rewrite on a turn the gateway marks as cache-expired. It holds observation_mask and relevance_filter by default; add command_digest to include it. A possible read-back still counts, and later turns replay the same digest.

Two cases keep the usual price. One is a digest the agent cannot fetch back. The other is a request where cache_optimizer places its own conversation breakpoint, because a new digest would hold that breakpoint back for the turn.

The list replaces the default, so a rule that adds a strategy restates the two defaults. An empty list turns the cold price off for every strategy it could name. coldTurn.writeCredit never credits command_digest. There is no benchmark for it yet, so scope it to a test with a rule.

Rule

{
  "when": { "metadata": { "experiment": ["cold-digest"] } },
  "coldTurn": { "premiumKinds": ["observation_mask", "relevance_filter", "command_digest"] }
}

code_graph

When a request carries several source files the agent read, code_graph keeps the functions in play and the functions they call. Other function bodies become a short marker, and their signatures stay. Hidden bodies stay retrievable.

It maps which symbols use which across the files, and matches that map against the current turn, not against the last file read. A file the agent is editing, or one the turn names by path, stays whole, so edits always see real bytes. Fresh tool results and retrieved originals are never outlined.

On cached history it outlines a file only when the saving beats the cache rewrite, retrieval is available, and the original is stored. Later turns replay the same outline, so the cache stays stable.

Source-file results from Read, cat -n, or sed -n stay whole until eight later assistant requests in that agent session. A later read or search of overlapping or adjacent lines also keeps the result whole. Set sourceReadMinAge per strategy to change the age floor; zero turns it off.

freshTailMinLines is off by default. To outline a whole-file source read on the request that first carries it, set a line count such as 150 and set sourceReadMinAge to zero. The age floor otherwise keeps first-sight reads whole. This lane saves the cache write and later reads of hidden bodies when the agent does not fetch them back.

The first-write saving uses the lowest proven rate in the edited message. Text beyond the last cache marker saves plain input; a 5-minute tail cannot claim the system prompt's 1-hour write rate.

coldTurn.premiumKinds works as it does for command_digest. With code_graph in the list, an outline of cached history is priced without the cache rewrite on a cache-expired turn. It is not in the list by default and has no benchmark yet.

Three things stay whole. A read that asked for a line range, because the agent wants those lines. A file the task names by path. And bodies of symbols the task names. Each outline the model fetches back makes the next one less likely, until the session makes no new ones.

It trades quality for cost

The agent asked for that file and gets signatures plus markers like [lines 1973-2036 · retrieve ctx_7c]. It has to call anyray_retrieve for a body it needs. Scope it to a test with a rule before you turn it on widely.

Rule params

{ "params": { "code_graph": { "freshTailMinLines": 150, "sourceReadMinAge": 0 } } }

relevance_filter

Keeps only the lines of a long tool output that matter to the latest message. It builds a query from that message, ranks every line, and keeps the top lines plus the first few for context. In a tool loop with no recent prose, it uses the first user message (the session task) and the latest tool-call arguments, including a call's description. Task matches take priority when the kept output reaches its size limit. When a recent query misses an output entirely, it can rank that output against the latest tool-call arguments. Removed lines stay retrievable, and the user's own words are never ranked or trimmed.

It edits older turns only, including the most recently completed one. It skips a source file that code_graph will outline, because line-by-line trimming would break the structure code_graph reads. Source-file tool results follow the same sourceReadMinAge and later-read checks described above.

It also reads assistant messages. Some agent frameworks, LangGraph among them, trim a tool result to a short placeholder and re-send the real output as an assistant message on the next step. Settled ones get filtered. One at the very end of the request is left whole, because the model has not read it yet.

semantic_cache

Off by default

When a request exactly repeats an earlier one, Anyray serves the stored response and skips the provider. A hit needs an exact match on the caller and on every field that can change the answer: model, system prompt, messages, tools, and sampling params.

A request that runs a guardrail or mutator is never served from it. It goes to the provider, so its guardrails check a real answer, and that answer is not stored for later requests.

It ships off because a coding agent never repeats a request: its transcript grows every turn. Over a month of fleet traffic it served 0.20% of requests, and none on Claude Code or Codex.

Turn it on for a workload that sends the same request again and again, such as a batch or SDK job with a fixed prompt.

context_dedupe

Off by default

Collapses a tool output the agent has already seen. A re-read file or a re-run test becomes a one-line marker, and a near-repeat keeps only the lines that changed. The first copy stays in full.

With request planning enabled, each collapse competes with other edits. The planner accounts for the copy that must stay in full when comparing the savings of different edits. Existing markers keep replaying unchanged when the planner declines new edits.

Error results are never collapsed, because a marker cannot say which call failed. A near-repeat is collapsed only when it rebuilds exactly from the first copy plus the diff, and saves tokens.

A client that can fetch originals back gets a retrieve ctx_… handle on a repeat or near-repeat marker. Other clients get the marker without it. Nothing is lost, since the first copy is still in the same request, and a handle the agent cannot resolve only makes it re-read the file.

A Read of part of a file the agent already read becomes a marker when every numbered line is in an earlier Read of that file, under the same line number. The marker names that Read and the line range. It never carries a handle, because that Read holds every line. Only the new read changes, and only when it is provably not cached yet. Set containedReads: false to turn this off. With firstSightOnly: true only this lane makes new edits, including on a client whose cache breakpoint is on the last message: a byte-identical re-read becomes the same marker, and other repeats and near-repeats stay whole. It stays held when the request uses Anthropic's request-level cache_control (automatic caching).

It also runs on API-key lanes. See API-key lanes.

context_compression

Off by default

The catch-all for bulky tool output. It minifies JSON and compresses known output shapes. When the context gets tight, it trims in whole units: paragraphs, log lines, diff hunks, or stack frames. JSON is minified on the turn the output arrives. Every other shrink waits until the output is history, so the model first reads each result byte for byte. Set firstSightOnly: true to keep only the arrival minify, and only when it removes whitespace and changes no other byte.

It never cuts code or JSON it cannot parse; that unit stays complete. It leaves JSON alone when minifying would change a value, such as a 64-bit database id. Removed parts stay retrievable.

It runs after the specialist strategies, so it never mangles their input or nests their markers. It stands aside when the history is cached, either anchored by cache_optimizer or cached automatically by the provider. It keeps running on uncached traffic.

output_externalize

Off by default

Replaces a whole bulky tool output with a short handle and stores the original. The agent pulls the exact bytes back when it needs them.

By default it applies to tool outputs of 4,000 characters or more, keeps the copy for one day, and never touches the current turn.

It runs only when the durable store can save the original and the client can fetch it back. It rewrites past messages, so it stands aside on prompt-cached traffic. That makes it a fit for uncached API traffic that carries one huge output, such as a SQL dump, on every request.

columnar_fold

On by default

A tool that returns a list of records repeats the same field names in every record. columnar_fold rewrites the list as one header row plus one row per record. A field with the same value on every record moves out once.

It reads tool results. For an app tool loop (clientTool of langgraph, langchain or sdk-agent) it does three more things:

• A nested object every record carries becomes dotted columns (tenant.name), so its field names are sent once too. A nested field some records lack still folds, and reads back as missing, not null.
• It reads JSON the app writes into its own assistant messages at the end of a request: a label line, then the JSON. An app that keeps tool results in state puts its data there. A reply the model wrote is never touched: an assistant message followed by a user turn or a tool result stays as it is, and so does one that is only JSON.
• A large JSON object one message repeats, such as a sub-schema an API spec repeats in every endpoint, is written once at the top of the message and each repeat becomes {"$ref":"#/$defs/s1"}. Only objects of at least 500 characters are shared, so a request parameter stays where the model reads it.

It drops nothing and stores nothing. Anyray unfolds the result and compares it to the original first, and the message passes through unchanged if anything differs. It also leaves alone anything the comparison cannot see: records with different keys, an object with a repeated key or a __proto__ key, and a number JSON cannot carry exactly (a 64-bit database id, or a decimal with more digits than a double holds). JSON followed by prose, and a message with more than one JSON body, are not folded by default. In an app tool loop their repeated objects are still shared.

It runs on messages of at least 400 characters with at least 4 records, and skips a fold that saves under 120 characters or under 5% of the message. It also leaves alone any table whose fold removes under 1,000 characters (repeated field names and constant columns, measured per table against the minified JSON so indentation does not count), so a short two-field list stays as the tool wrote it. For a fixed configuration the same input always folds to the same bytes.

It runs on every request. A result is folded the first time it arrives, so it is smaller while it is still fresh. When the result goes stale, observation_mask can still replace it with a retrieval marker at its usual point, priced by the folded size. A retrieve returns the original result, not the folded one. A session that was already running when the fold reached it folds only the results that arrive after, so a cached result never changes.

Setting	Effect
enabled: false	Off everywhere.
default (params.scope: "all")	Every request, on any lane.
params.scope: "app_harness_no_retrieve"	Only app tool loops (clientTool of langgraph, langchain or sdk-agent) on a lane that cannot retrieve, decided once per session.
params.appStateMessages: false	App tool loops fold tool results only, not JSON the app wrote into assistant messages.
params.flattenNested: false	App tool loops keep nested objects as cells.
params.shareRepeatedObjects: false	App tool loops send every repeated object in full.
params.embeddedJson: true	Also a message with several JSON bodies or text after its JSON. Each body folds and the text around it stays as written. A message with an Anyray marker outside its JSON still stays whole.

Changing its thresholds can refold results that are already cached, once, on a session the gateway does not pin.

repeat_factor

On by default

Long blocks of text repeat across tool results and pasted user context: the same licence header on nine file reads, the same schema in three query results, or a quoted block carried into a follow-up. repeat_factor keeps the first copy and replaces each later copy with a one-line marker.

Nothing is stored. The block the marker points to is still in the request, so the model reads it there instead of calling a tool. It never rewrites the first copy, or a span another strategy already changed.

A block must be at least 8 lines and 400 characters. It scans up to 20,000 lines per request. By default it scans tool, function and user messages. Assistant messages stay unchanged.

tool_pruning

Off by default

Drops tools the model is unlikely to call, which cuts schema tokens on eligible requests. When in doubt, it keeps the tool. A tool stays if its name or description shares a topic word with the conversation. Generic words like "get" or "find" do not count, and neither do words most descriptions share. MCP tools (mcp__…) are never pruned.

If the catalog has core file or shell tools such as Read, Grep, or Bash, or a terminal tool such as finish, Anyray keeps every tool until the agent calls one. Small catalogs stay intact. In larger ones, core tools, terminal tools, and tools already called stay; unrelated ones can go.

A new tool set every turn would change the tools list and break the prompt cache. So when Anyray can pin decisions for the session, it decides once and replays the same set. It decides again when the tool list changes, when tool_choice forces a pruned tool, or after the model calls a pruned tool. Without pins it stands aside on cached traffic. Turn it on for a fixed tool catalog; a list that changes often keeps resetting it.

prompt_compression

Off by default

Cleans filler out of system and user messages over 400 characters. It collapses repeated spaces and blank lines, and drops exact duplicate sentences and paragraphs. The first copy stays.

It never rewords text, and it keeps line breaks and list structure. Code passes through byte for byte, and tool results are never touched.

It also runs on API-key lanes. See API-key lanes.

tool_schema_compression

Off by default

Shrinks each tool's description. It collapses whitespace, strips openers like "This tool allows you to…", and cuts filler. Types, enums, and required fields stay byte for byte, so every tool stays callable. Parameter descriptions shrink too, but details that tell the model what to pass, such as "for example PAY or CORE", stay.

It reads only each tool's own bytes, so the same schema always gives the same output. It pairs with tool_pruning: pruning drops tools, and this shrinks the ones that stay.

On ten verbose MCP schemas the default pass trims about 8% of the request. An optional cap on description length takes that to about 32%.

client_prefix_compression

Off by default

Some clients send a large fixed prefix on every request. Claude Code's system prompt and built-in tool descriptions run to about 12k tokens. This strategy holds a hand-written table: for each known passage, the exact bytes a client version sends and a shorter form that keeps every instruction. A passage must match byte for byte, so anything else is forwarded unchanged.

The table covers Claude Code 2.1.257, 2.1.260 and 2.1.282 on Sonnet 5. By default it shortens the system prompt only. It can also shorten the built-in tool descriptions. Subagent prompts are left unchanged.

It saves about 20% of input per turn. On a real repository, though, sessions got longer, and one long session costs as much as four or five short ones. Compare your own sessions against a direct connection before you turn it on.

It only runs when the prefix is not cached. Claude Code's prefix is the same in every session, so the provider usually has it cached, and rewriting it made sessions cost 1.25x to 1.72x a direct connection. When the system prompt is cached, the strategy stands aside. If only the tools are cached, it shortens the system prompt and leaves the tools. A warm_prefix guard reason in the result header marks this. You can set it to rewrite a cached prefix anyway, which pays back only when enough sessions share one cache lifetime. Use that to measure on your own traffic.

param_tuning

Off by default

Caps an over-large max_tokens at 65,536 by default. A request already under the cap passes through unchanged.

This is a guardrail against runaway replies, not a way to save money. Providers bill the tokens the model actually generates, so a lower ceiling does not lower the bill. A cap set too low can cut a reply short.

provider_context_trim

Off by default

On a large Claude request, in any billing mode, it adds Anthropic's own context-management edit (clear_tool_uses_20250919) and its beta header. Anthropic then clears old tool results before billing. API-key requests stop paying for those tokens, and subscription seats stop spending their usage limit on them.

Anyray sets the edit so Anthropic clears only when the window runs low, keeps the tool calls of the active exchanges, and clears just enough to restore room.

It runs before Anyray's own tool-output strategies. They leave the results Anthropic will clear alone, and can still shrink the ones it keeps.

reasoning_budget

Off by default

It lowers reasoning effort on metered reasoning models. It never raises a value the client set.

• Anthropic output_config.effort. Decided once per session. On the session's first request it lowers a client value above the target, or sets the target on allow-listed Claude models, and it keeps that value on every later turn. A session it first sees mid-flight keeps the client's effort. Either way the session's thinking budget is left alone: one reasoning decision per session.
• Anthropic thinking.budget_tokens. Capped on a routine tool-resume turn (the new input is tool results, and none failed). A harness reminder after the tool results, such as Claude Code's token-budget message, does not count as new input. It never adds or removes thinking.
• OpenAI. On a routine tool-resume turn it sets or lowers GPT-5 verbosity and reasoning_effort, and backs off after a tool error.

On a pinned session the first decision is saved, so later turns reuse the same value, and a model switch resets it. It reports zero estimated savings: a lower effort is a setting the client can choose too, not a saving against a direct run at the same effort. Run it as a canary and measure output tokens and session length.

Effort is part of Claude's prompt-cache key

Changing output_config.effort between turns makes the request read nothing from cache, tools and system included. That is why the effort is held for the whole session, also when the optimizer is skipped, and why the strategy never changes it without the gateway's pin store. There is no per-turn effort scope, so an anthropicEffortScope param is ignored.

output_shaping

Off by default

On a routine tool-resume turn, it adds one short line to the end of the request. The line asks the model to point at file paths and diffs instead of repeating context. It goes after everything already in the request (a new text block, or appended to a last message that is a plain string), so no earlier text changes and the cached prefix stays the same. It stands aside when that string is protected.

It waits for 2 assistant turns, skips a turn with a failed tool result, and never runs on subscription seats.

It reports zero estimated savings, since the line guides the model but does not cap it. Measure the change in output tokens instead. Some coding agents flag instructions inside content as prompt injection, so try it on one lane first.

thinking_trim

Off by default

Agents re-send past thinking blocks on every turn, and you pay for them each time. thinking_trim drops a past thinking block once that turn's visible output already covers it.

It only trims reasoning from finished user turns. The tool loop the model is working through keeps all its thinking. From finished turns it keeps the opening plan of each tool loop, the reasoning after a tool error, and the reasoning on a turn that edits a file. To lower the current turn's thinking, use reasoning_budget.

Rewriting reasoning the provider has already cached costs the cache-write price (about 1.25x input instead of 0.1x) and raised input cost 40-65% on replayed sessions. So it trims where nothing is cached yet, and rewrites cached reasoning once only when the trim pays back over the rest of the session. Trims it holds show as a warm_prefix decline on the trace.

A turn ends when you send the next message. Messages Claude Code adds on its own do not end it: a loaded skill's instructions, Tool loaded., a Stop hook's feedback, a background-task notification.

An opt-in setting also trims inside the live tool loop. On Sonnet 5 with adaptive thinking, that made the model redo the reasoning it lost, costing about 1.3x the output tokens and more than no trimming. Turn it on only where an audited holdout shows a net saving.

Claude Fable and Mythos 5.1, Opus 5.5, Sonnet 5.5 and newer check every re-sent thinking block against the exact prompt it was written for. The only removal they accept is dropping a block the first time it is sent, which is always live-loop thinking. With default settings thinking_trim saves nothing on those models.

observation_mask, relevance_filter and code_graph do edit history on those models by default. On the first-party API the provider then drops every thinking block after the edit and answers without that reasoning; Bedrock keeps them. The org setting protectThinkingBinding (PUT /admin/optimizer/settings) holds those edits too, as a thinking_binding decline.

window_budget

Off by default

When a session's history outgrows the model's context window, window_budget removes middle messages and leaves a retrievable placeholder in each spot. It scores each message by how relevant it is to the live turn and how recent it is.

It never removes the task setup (the first message), the live turn, or system messages. The agent can fetch every removed message back.

It ships off because it rewrites the middle of the history. Most agents compact their own context; turn it on for a client that does not.

cache_optimizer

Providers give the cache discount only when a request starts with the same bytes as the last one. cache_optimizer keeps that start stable: it sorts tools into a fixed order, adds Anthropic cache breakpoints, and sets a prompt_cache_key on OpenAI models and on Mistral routes. It leaves the tool order alone for app frameworks (langgraph, langchain, sdk-agent), whose code already sends the same order every turn. Set sortTools: false to stop sorting for everyone. On metered markerless Claude sessions, it can also mark the end of the latest message once the full prompt clears the model's cache minimum. The gateway's verified session history must show a follow-up inside the five-minute cache window.

It never edits prompt text. If your client already places its own cache breakpoints, it stands aside and reports client_managed. Other no-action guards report fixed reasons through strategyDeclines in the optimizer protocol. It runs last, so it sees the final request after every other strategy.