A small, self-contained transform that runs inside the optimizer's pipeline.
A strategy is the unit of optimization. It inspects one request, transforms it or leaves it
untouched, and records what it did (kind, short summary, token/cost estimates). It applies only
when its own condition matches, and one that throws is skipped, so a broken strategy can never
break inference.
Some strategies leave a handle so the model can fetch the original with anyray_retrieve.
Others change its representation while keeping the values inline. Accepted edits replay on later
requests to keep the prompt cache stable (how). The examples show
the available transforms; the reference lists which are on
by default and the checks that can hold them back.
Replaces an old, large tool result with a one-line marker, so you stop paying for it every turn.
The agent can fetch the original back. It never masks a failed tool call, a source file the agent
read, a file it is editing, or the latest exchange.
Next request
userAdd a tax line to the checkout total.
turn 4grep -rn "total" src/ → 212 matchesstale: used in turn 5[anyray: observation masked · 18432 chars · retrieve ctx_9f2k via anyray_retrieve]
turn 9npm test → 2 failed, 126 passedkept: failed call
turn 12curl localhost:3000/api/orders → 500 ordersstale: used in turn 13[anyray: observation masked · 51200 chars · retrieve ctx_c41e via anyray_retrieve]
turn 15Read src/checkout.ts → 380 lineskept: file being edited
turn 22npm test → 128 passedkept: current turn
turn 23git diff --stat → 2 files changed
userAdd a tax line to the checkout total.
turn 4grep -rn "total" src/ → 212 matches[anyray: observation masked · 18432 chars · retrieve ctx_9f2k via anyray_retrieve]same bytes as last request
turn 9npm test → 2 failed, 126 passed
turn 12curl localhost:3000/api/orders → 500 orders[anyray: observation masked · 51200 chars · retrieve ctx_c41e via anyray_retrieve]same bytes as last request
turn 15Read src/checkout.ts → 380 lines
turn 22npm test → 128 passed
turn 23git diff --stat → 2 files changed
in cachelocked
✓First request: nothing is cached yet
✓5 rows written to the cache
✓1 row read from the cache
✓One-time rewrite: 4 rows written to the cache again, once
✓6 rows read from the cache, Anyray's edit included · 0 rows cached again
✓6 rows read from the cache, Anyray's edit included · 0 rows cached again
Old tool outputs the agent is done with still go out on every request.The provider has cached these rows, so they are locked.
Near the context limit, old tool output becomes a marker. Failures and edited files stay.This rewrites the cache once, and only because the saving pays for it.
The markers go out as the same bytes, so the provider reads them from the cache.If the agent needs an output again, it fetches it by handle.
The markers go out as the same bytes, so the provider reads them from the cache.If the agent needs an output again, it fetches it by handle.
By default, folds exact repeated blocks in fresh test output into references to the first copy,
which stays inline. The optional full digest keeps test failures and counts, groups grep output
by file, and collapses repeated log lines. That lane keeps the full output retrievable.
Next request
assistantbash: pytest -v
tool============ test session starts ============not needed: bannertests/test_api.py::test_list_orders PASSED [ 0%]not needed: passing test… 1,241 more linesnot needed: progressFAILED tests/test_auth.py::test_login_lockout - assert False is Truekept: failureFAILED tests/test_billing.py::test_refund_rounding - assert 999 == 1000kept: failure====== 2 failed, 1238 passed in 41.20s ======kept: count line… digest — full output retrievable (retrieve ctx_a4f1)
assistantTwo failures. read_file: src/auth.py
tool41 return user.failed_logins > MAX_ATTEMPTS
assistantedit_file: src/auth.py
toolsrc/auth.py: line 41 changed
assistantbash: pytest tests/test_auth.py
assistantbash: pytest -v
tool============ test session starts ============tests/test_api.py::test_list_orders PASSED [ 0%]… 1,241 more linesFAILED tests/test_auth.py::test_login_lockout - assert False is Truekept: failureFAILED tests/test_billing.py::test_refund_rounding - assert 999 == 1000kept: failure====== 2 failed, 1238 passed in 41.20s ======kept: count line… digest — full output retrievable (retrieve ctx_a4f1)same bytes as last request
assistantTwo failures. read_file: src/auth.py
tool41 return user.failed_logins > MAX_ATTEMPTS
assistantedit_file: src/auth.py
toolsrc/auth.py: line 41 changed
assistantbash: pytest tests/test_auth.py
in cachelocked
✓First request: nothing is cached yet
✓4 rows written to the cache
✓1 row read from the cache
✓One-time rewrite: 3 rows written to the cache again, once
✓6 rows read from the cache, Anyray's edit included · 0 rows cached again
✓6 rows read from the cache, Anyray's edit included · 0 rows cached again
The log went through whole. This client caches each request to its end, the log included.The model is still reading the log, so it stays as is.
One exchange later the log has settled. It keeps the failures and the count line.This rewrites the cache once, and only because the saving pays for it.
The digest is pinned, so the provider reads the same bytes from the cache.The full log stays retrievable.
The digest is pinned, so the provider reads the same bytes from the cache.The full log stays retrievable.
Shortens a test log the agent reads. When a test command in the session wrote a log to a file, the
agent's Read of that file returns its summary and its failing tests, each kept whole at its own
line number. A passing run returns its results. A marker names every omitted range as a Read
offset and limit, so the file is the way back. Reading the whole log again, or a range a marker
named, returns the file whole, and so does a log that changed since its run or has nothing to cut.
Each time it reads a log whole, it records why.
It runs in Connect, only for Claude Code, and is on by default. It is not an optimizer pipeline
step: turn it off for the org on Console → Optimizer, or for one team or user in Tool
policy (setting). Its savings show under
log_read. Each seat reports whether the lane is active, off by policy (and at which level), or not
available on its client, so the console can count them. A regret guard, the one the optimizer's
strategies have, watches how often the agent reads back lines a digest omitted. If too many are
read back, the gateway holds the lane back for the org for an hour and Connect records the reason
guarded. Turn on Holdout in the Measurement card and pick Test log reads to hold out a share of
sessions from the lane and read the saving off cost per session, over the sessions that read a test
log (how it is counted).
Keeps the lines of a long tool output that match the task or the latest message, plus the first
few for context. Removed lines stay retrievable, and a person's own prose stays word
for word.
Next request
…… 8 earlier requests, settled
assistantfetch_logs(tier: "web")
tool10.0.2.14 GET /health 200 2mskept: first 3 lines10.0.2.14 GET /assets/app.js 200 5ms10.0.2.14 GET /home 200 8ms… 312 more linesno match for the ask… [312 line(s) omitted by anyray · retrieve ctx_5a]10.0.9.3 POST /checkout 500 812mskept: matches the ask10.0.9.7 POST /checkout 500 903mskept: matches the ask… 883 more linesno match for the ask… [883 line(s) omitted by anyray · retrieve ctx_5a]
assistantPulled last night's access log.
userFind the 500 errors on /checkout.kept: the user's words
assistantBoth 500s are POST /checkout, over 800 ms.
userDid they start after the 02:00 deploy?
…… 8 earlier requests, settled
assistantfetch_logs(tier: "web")
tool10.0.2.14 GET /health 200 2mskept: first 3 lines10.0.2.14 GET /assets/app.js 200 5ms10.0.2.14 GET /home 200 8ms… 312 more lines… [312 line(s) omitted by anyray · retrieve ctx_5a]10.0.9.3 POST /checkout 500 812mskept: matches the ask10.0.9.7 POST /checkout 500 903mskept: matches the ask… 883 more lines… [883 line(s) omitted by anyray · retrieve ctx_5a]same bytes as last request
assistantPulled last night's access log.
userFind the 500 errors on /checkout.
assistantBoth 500s are POST /checkout, over 800 ms.
userDid they start after the 02:00 deploy?
in cachelocked
✓First request: nothing is cached yet
✓3 rows written to the cache
✓2 rows read from the cache
✓One-time rewrite: 1 row written to the cache again, once
✓5 rows read from the cache, Anyray's edit included · 0 rows cached again
✓5 rows read from the cache, Anyray's edit included · 0 rows cached again
The model has not read the access log yet, so it goes through whole.This client caches each request to its end, so the log is cached as it arrives.
The user asks about /checkout 500s. It keeps the log lines that match the ask.Only the log is rewritten in the cache, once, because the saving pays for it.
The filtered log is pinned, so the provider reads the same bytes from the cache.It edits cached rows only once some history has settled.
The filtered log is pinned, so the provider reads the same bytes from the cache.It edits cached rows only once some history has settled.
When a request carries several source files, it keeps the functions in play and the functions they
call. Other function bodies become a short retrievable marker under their signature. Reads stay
whole until they are eight requests old, and a file the agent is editing stays whole.
Next request
checkout.tscharge() and refund(), wholekept: named in the ask
pricing.tsprice(), wholekept: named in the ask
tax.tscalcTax(), wholekept: price() calls it
loyalty.tsexport function award(order: Order): number {kept: signature … 58 body linesnot linked to the ask … [58 lines · retrieve ctx_7c]}… 13 more function bodiesnot linked to the ask… 13 more signatures, each body one marker
…… 9 requests since these reads
userrefund() pays back more than charge() took. Is price() the bug?
assistantprice() rounds before calcTax() adds tax. Edit: pricing.ts
checkout.tscharge() and refund(), whole
pricing.tsprice(), whole
tax.tscalcTax(), whole
loyalty.tsexport function award(order: Order): number {kept: signature … 58 body lines … [58 lines · retrieve ctx_7c]}… 13 more function bodies… 13 more signatures, each body one markersame bytes as last request
…… 9 requests since these reads
userrefund() pays back more than charge() took. Is price() the bug?
assistantprice() rounds before calcTax() adds tax. Edit: pricing.ts
in cachelocked
✓First request: nothing is cached yet
✓5 rows written to the cache
✓3 rows read from the cache
✓One-time rewrite: 2 rows written to the cache again, once
✓6 rows read from the cache, Anyray's edit included · 0 rows cached again
✓6 rows read from the cache, Anyray's edit included · 0 rows cached again
The agent read four files 9 requests ago. It maps which functions call which.A read stays whole for eight requests. These rows are cached, so they are locked.
The ask names refund(), charge() and price(). loyalty.ts is linked to none of them.Its bodies become markers in a one-time rewrite, made because the saving pays for it.
The outline goes out as the same bytes, so the provider reads it from the cache.If the agent needs a body, it retrieves it by handle.
The outline goes out as the same bytes, so the provider reads it from the cache.If the agent needs a body, it retrieves it by handle.
Sends a list of records as one header row plus one row per record, so the field names appear once.
Anyray unfolds the result and checks it matches the original first. It is on by default.
An app-state API spec can also lose response-schema descriptions and examples, which cannot be
retrieved. Scope and controls.
tool{"__anyray_columns__":["object","id","amount","status"],field names, once"__anyray_rows__":[[{"object":"order","id":"ord_01","amount":1995,"status":"paid"},field names repeat{"object":"order","id":"ord_02","amount":4200,"status":"failed"},"object" stays in the row… 498 more records]values stay, in order
tool{"__anyray_columns__":["object","id","amount","status"],field names, once"__anyray_rows__":[["order","ord_01",1995,"paid"],["order","ord_02",4200,"failed"],… 498 more rows]}same bytes as last request
assistantPaid orders this week total $48,210.40.
in cachelocked
✓First request: nothing is cached yet
✓1 row written to the cache
✓1 row read from the cache · 0 rows cached again
✓3 rows read from the cache, Anyray's edit included · 0 rows cached again
✓3 rows read from the cache, Anyray's edit included · 0 rows cached again
The question goes out first.The provider caches it, so it is locked. The model calls list_orders.
The list is new, so it is editable. Every record has the same field names.The names go into one header. Every value stays, checked against the original first.
The folded list is cached now. Anyray sends the same bytes, so it reads from the cache.
The folded list is cached now. Anyray sends the same bytes, so it reads from the cache.
Keeps the first copy of a long block that repeats across tool results, such as a license header
on every file read, and points each later copy at it. The first copy stays in the request. It is
on by default.
Next request
billing.ts// SPDX-License-Identifier: MITkept: first copy… 61 more lines: license text and imports
checkout.ts// SPDX-License-Identifier: MITrepeat of billing.ts… 61 more lines, the same as billing.tsrepeat of billing.ts[anyray: 62 identical lines elided — repeat of the 62-line block starting "// SPDX-License-Identifier: MIT" earlier in this conversation]export async function placeOrder(cart: Cart) {kept: its own code
assistantEdit: src/checkout.ts
billing.ts// SPDX-License-Identifier: MIT… 61 more lines: license text and imports
checkout.ts// SPDX-License-Identifier: MIT… 61 more lines, the same as billing.ts[anyray: 62 identical lines elided — repeat of the 62-line block starting "// SPDX-License-Identifier: MIT" earlier in this conversation]export async function placeOrder(cart: Cart) {kept: its own codesame bytes as last request
assistantEdit: src/checkout.ts
in cachelocked
✓First request: nothing is cached yet
✓1 row written to the cache
✓1 row read from the cache · 0 rows cached again
✓2 rows read from the cache, Anyray's edit included · 0 rows cached again
✓2 rows read from the cache, Anyray's edit included · 0 rows cached again
The agent reads billing.ts. It opens with a 62-line license header.The provider caches the read, so it is locked.
checkout.ts is new, so it is editable. It opens with the same 62 lines.The repeat points at billing.ts, which stays whole for the model to read.
The pointer is cached now. Anyray sends the same bytes, so it reads from the cache.
The pointer is cached now. Anyray sends the same bytes, so it reads from the cache.
Off by default. After a new message, drops a finished turn's reasoning past its opening plan.
Turns that edit files or recover from an error keep their reasoning. Cached reasoning stays unless
a one-time rewrite
pays back. On Opus 5.5, Sonnet 5.5, Fable and Mythos 5.1 and newer it trims nothing
with default settings.
Next request
userRename getUser to fetchUser.
assistantthinking: Plan: find every call site first.kept: opening plantool_use: grep("getUser")
…… 3 edits, each with its reasoningkept: edit turns
assistantthinking: All 3 files use fetchUser now. Report it.past the opening planRenamed getUser to fetchUser in 3 files.
userNow run the tests.closes the last turn
assistantthinking: Run the full suite to check the rename.tool_use: bash("npm test")
userRename getUser to fetchUser.
assistantthinking: Plan: find every call site first.tool_use: grep("getUser")
…… 3 edits, each with its reasoning
assistantthinking: All 3 files use fetchUser now. Report it.Renamed getUser to fetchUser in 3 files.same bytes as last request
userNow run the tests.
assistantthinking: Run the full suite to check the rename.tool_use: bash("npm test")
in cachelocked
✓First request: nothing is cached yet
✓3 rows written to the cache
✓3 rows read from the cache · 0 rows cached again
✓5 rows read from the cache, Anyray's edit included · 0 rows cached again
✓5 rows read from the cache, Anyray's edit included · 0 rows cached again
The agent plans, then makes 3 edits, each with its reasoning.The provider caches those requests, so they are locked.
The new message closes the last turn. Its final reasoning is not cached yet.That reasoning goes before the provider caches it. The plan and edits keep theirs.
The trimmed reply was cached as sent, so it repeats in the same bytes.Reasoning in the tool loop the agent is working through is never trimmed.
The trimmed reply was cached as sent, so it repeats in the same bytes.Reasoning in the tool loop the agent is working through is never trimmed.
Off by default. Shortens each tool's description by cutting stock openers and filler. Types, enums,
and required fields stay byte for byte, so every tool stays callable.
Request 2
toolsearch_kb "This tool allows you to search the knowledge base."stock opener query "The text that you would like to search for."filler phrase
toolcreate_ticket "This tool allows you to create a support ticket.stock opener Please be sure to fill every field."kept: an instruction priority enum ["P0", "P1", "P2"], requiredkept: schema as is
userFind our refund policy for annual plans, then file a P1.
toolsearch_kb "search the knowledge base." query "The text to search for."same bytes as last request
toolcreate_ticket "create a support ticket. Please be sure to fill every field."kept: an instruction priority enum ["P0", "P1", "P2"], requiredkept: schema as issame bytes as last request
userFind our refund policy for annual plans, then file a P1.
Keeps the start of each request stable so the provider's cache discount applies. It sorts tools
into a fixed order (except for app frameworks, which already send one) and, on metered Claude traffic, adds cache breakpoints once a request follows
the last within five minutes. It never edits prompt text. It stands aside when the client sets its own cache markers.
Request 3
tools[search, book_hotel, get_weather] client order, varies by turnorder changes per turn"cache_control":{"type":"ephemeral"} on search, the last tool
systemYou are a travel agent. Quote prices in the guest's currency.kept: text never edited"cache_control":{"type":"ephemeral"} on the last system block
userFind a hotel near the venue for May 3 to 5.
assistantsearch(query: "hotels near the venue, May 3 to 5")
result12 hotels found, prices in EUR"cache_control":{"type":"ephemeral"} on the latest message
resultBooked: Hotel Lido, May 3 to 5"cache_control":{"type":"ephemeral"} on the newest message
tools[book_hotel, get_weather, search] name order, same every turn"cache_control":{"type":"ephemeral"} on search, the last toolsame bytes as last request
systemYou are a travel agent. Quote prices in the guest's currency.kept: text never edited"cache_control":{"type":"ephemeral"} on the last system blocksame bytes as last request
userFind a hotel near the venue for May 3 to 5.
assistantsearch(query: "hotels near the venue, May 3 to 5")
result12 hotels found, prices in EUR"cache_control":{"type":"ephemeral"} on the latest messagesame bytes as last request
resultBooked: Hotel Lido, May 3 to 5"cache_control":{"type":"ephemeral"} on the newest messagechanged by Anyray
in cachelocked
✓Request 1 had no marker, so this request writes the cache first
✓5 rows read from the cache, Anyray's edit included · 0 rows cached again
✓5 rows read from the cache, Anyray's edit included · 0 rows cached again
Request 2 comes inside the cache's five minutes, so a cache write pays off.The tools go in name order, as on Request 1, and three markers are added.
The client sends the tools in another order. Anyray sorts them back, so they match.The latest-message marker moves to the newest row, so the new rows are cached too.
The client sends the tools in another order. Anyray sorts them back, so they match.The latest-message marker moves to the newest row, so the new rows are cached too.
Off by default. When the same caller sends an exact repeat of a recent request, Anyray returns
the stored answer and skips the provider. A reworded question is not a repeat, and a streamed
request, or one that runs guardrails, is never served from it.
Requests 3 and 4
request 1"What is our refund policy?"→ a miss: sent to the provider, answer stored
request 2"What is our refund policy?"→ stored answer returned, no provider callrepeat of request 1
request 3"What's the refund policy?"→ reworded, so not a repeat: sent to the provider
request 4"What is our refund policy?" · stream: true→ streamed: sent to the provider
request 1"What is our refund policy?"→ a miss: sent to the provider, answer stored
request 2"What is our refund policy?"→ stored answer returned, no provider call
request 3"What's the refund policy?"→ reworded, so not a repeat: sent to the provider
request 4"What is our refund policy?" · stream: true→ streamed: sent to the provider
✓A miss: the provider answers as usual, and Anyray stores the answer
✓A hit: no provider call, so nothing is billed
✓Not repeats: both go to the provider unchanged
•Not repeats: both go to the provider unchanged
An SDK job asks a question for the first time, so it goes to the provider.Anyray stores the answer for this caller, for up to an hour.
The job sends the exact same request again, from the same caller.Anyray returns the stored answer. The request itself is never edited.
A reworded question is a different request, so it goes to the provider.A streamed request is never served from the store either.
A reworded question is a different request, so it goes to the provider.A streamed request is never served from the store either.