Benchmarks

Reproducible numbers: the real optimizer over fixed synthetic payloads.

These are isolated request measurements: transformation efficiency, not total session spend (provider prompt-cache writes, persistent client transcripts, retrieval turns, and tool reruns can dominate the bill, and session-level regressions are monitored separately). The page also measures compression, not answer quality; how Anyray proves it isn't degrading sessions, and how you verify that on your own traffic, is Guardrails. Every number here is reproducible from the benchmarks repository.

Benchmarks

How it's measured

Each workload runs twice: a control baseline and an optimized run with the workload's strategy pinned at one knob. A straight before/after. The harness drives a live optimizer over its HTTP contract (pin the strategy via /admin/optimizer/settings, transform through /v1/optimize, read back the result); no optimizer internals are vendored. Token counts are real BPE tokens (gpt-tokenizer o200k_base, the GPT-4o/4.1 family), measured the same way for both runs.

Patterns

The common token-waste patterns users and coding agents fall into every day. These are what Anyray benchmarks against.

Waste pattern	Why it's costly
Paste a whole log or data dump and ask one question	the single most common thing engineers do with an assistant, billed in full every turn
Agents re-read entire files when the question needs signatures	~70% of a coding agent's tokens are irrelevant file reads
Tool-schema bloat rides along every call	55k+ tokens of tool definitions before the first message
RAG over-retrieval	3–5× more chunks fetched than the answer uses
Agents resend the whole history every turn	the #1 pattern: 4–15× chat tokens; a 50-turn session bills ~25:1 input:output
Redundant / near-duplicate requests	40–60% of enterprise traffic is repetitive (18% exact, ~47% similar)

Headline savings

Whole-request input-token reduction, measured by running each workload's strategy through a live optimizer at a fixed knob, grouped into suites:

Suite	Workloads	Before (tok)	After (tok)	Saved
Logs & structured data	3	126,998	18,383	86%
Code context	5	15,161	7,745	49%
Tools & retrieval	3	12,953	3,354	74%
Agent operations	3	89,013	24,033	73%
Cross-session memory recall	5	18,809	3,180	83%

Run it yourself

The suite needs a reachable optimizer (the Anyray stack brings one up in-network on :8088) and its admin token. Then:

Run the benchmarks

git clone https://github.com/anyrayHQ/benchmarks && cd benchmarks
cp .env.example .env          # set ANYRAY_OPTIMIZER_URL + ANYRAY_ADMIN_TOKEN
./run.sh --all                # writes <suite>/results/*.json

Each suite writes results/control.json and results/optimized.json, the committed real scores. Full per-workload tables and a live-provider cross-check are in the repo's RESULTS.md.