Data boundary
Content stays in your environment. Only metadata leaves.
Billing, connectors, spend rows, and logs are always metadata only. Trace content follows the content mode.
Encryption at rest
Content is stored as AES-256-GCM with ANYRAY_CONTENT_KEY (openssl rand -hex 32). No valid key,
no boot. No Anyray surface shows it decrypted: the trace API returns ciphertext. The one reader of captured traffic is
the credential scrub, which decrypts rows inside the gateway
and returns key ids, counts and connector hosts, never content. Switching to plaintext later does not unlock stored
content.
Org artifacts are decrypted only for the people they are shared with. An artifact is a page someone
published from a Claude session (POST /connect/artifacts), not captured
traffic. Publishing is on by default. An admin turns it off with
artifactsEnabled, or stops sharing with the
whole organization with artifactsOrgShare.
Content mode & fail-safe
One org-wide mode: Content privacy page or PUT /admin/settings. Audit-logged.
| Mode | Behavior |
|---|---|
encrypted (default) | Ciphertext. No surface reads it back |
off | No content stored |
plaintext | Readable in the console. Needs ANYRAY_ALLOW_PLAINTEXT=true |
Metadata stays readable in every mode. Without the gate, plaintext falls back to encrypted (key
set) or off. Removing the gate affects new captures only. ANYRAY_CONTENT_MODE governs only BYO
/v1/record and attach mode.
Content-free spend store
One metadata row per request, kept per ANYRAY_SPEND_RETENTION_DAYS (unset = forever).
| Field | Detail |
|---|---|
| Who | User and team, from x-anyray-metadata |
| Tool | Fixed id (claude-code, codex), never the User-Agent |
| Device | SHA-256 of the enrollment public key of a Connect-enrolled machine, taken from the verified key, never from a header |
| What | Model, provider. A model not shaped like a model id is stored as the id inside it that prices the same (my deployment/claude-sonnet-5-5 is stored as claude-sonnet-5-5), else unknown. On a Bedrock row, whether it ran on a regional or global inference profile when the gateway can tell, never the profile id or ARN |
| Route | Path only (/v1/messages). Query dropped, seat credential masked, bad segment :invalid, custom host flagged as via_custom_route |
| Session | Client id as sent (x-claude-code-session-id, prompt_cache_key; send an opaque one), subagents as <session>|agent|<agent id>, else a 32-char digest. A Connect hook row keeps the session id its hook sent, in its own column |
| Numbers | Tokens and cache split, cost, savings, latency, status, request size. On a Claude or Codex seat row, the share of the seat's usage limit the provider reported (0 to 1) |
The derived session id is an unkeyed hash of the system prompt and first turn. It cannot recover the prompt, but can confirm a guessed one.
Other local tables
| Table | Holds | Kept |
|---|---|---|
anyray_auth_events | Auth and cap rejections for Diagnose | 30 days |
anyray_batch_read_claims | One digest of tenant and tool-call id per credited batch read, so each is credited once. No user, path or text. Erase endpoint does not apply | 7 days |
anyray_client_test_results | Client test results: plan hashes, versions, token-shaped ids, counts, ratios, closed verdict enums and receive times. No task text, repo URLs or user identities | Until removed from the database |
anyray_cohort_triggers | For a measured Test log reads holdout: tenant, strategy, the Claude Code session id, whether the lane served or held back its digest, and when. No user, path or text. Erase endpoint does not apply | 100 days, or ANYRAY_SPEND_RETENTION_DAYS if shorter |
anyray_gdpr_erase_seat_hashes | Seat pseudonyms each GDPR erase reached, by random erase id, so other replicas and a retry reach the same seats. No address | 7 days |
anyray_health_lifecycles | One row per run of a replica for Health: start, last report, clean stop, and its latest memory, delay and pool numbers. No user, path or text | 7 days after the run ends |
anyray_health_problems | Health problems by fixed id: when each opened and closed, the replica, and the measured number | 90 days after closing |
anyray_health_samples | One row per replica every 15 minutes for Health (CPU, memory, event-loop delay, pool and request counts), plus one database reading per 15 minutes (size, server connections, transaction and lock counts, recovery and autovacuum state, the Postgres version number, and up to five allowlisted Anyray table names with size and row counts) | 7 days |
anyray_hook_arrival_candidates | Daily counters of ranged trims only the arrival check would admit: tenant, UTC day, pseudonymized command, tool, stage, size bucket, count, summed predicted net, and how many used the measured horizon. No seat, path or output. Erase endpoint does not apply | 30 days |
anyray_hook_tee_bets | Hook trims and read-backs: pseudonymized seat and command, tool, lane (digest/ranged), model, counters, and on a ranged trim the two ranged-check verdicts (true or false, two predicted numbers, two reason names). The tee id is salted on the developer's machine. No path or output. The erase endpoint removes the user's rows (in its own tenant on a tenant-scoped request) | 30 days |
anyray_replica_health | One health snapshot per replica for Health | 15-min refresh, 7 days |
anyray_spend_connector_events | Spend connector metadata. Owner GDPR endpoints cover it | 90 days or ANYRAY_SPEND_RETENTION_DAYS |
What the Billing App rollup sends
Every 15 minutes and on health changes. Content privacy → Shared shows the last payload.
| Part | Contents |
|---|---|
| Seats | Active count, salted seat hashes, attributed-user count. No identities |
| Usage | Requests, tokens, cost, savings by model and provider |
| Prompt cache by user | Per day and salted user pseudonym, always sent: the fields below. Top 50 users by request cost; the rest summed into one * row. Service and bot accounts excluded. No user ids, no request cost |
| Prompt cache warmth by user | For the same top 50 pseudonyms: per UTC hour, the minutes the cache was warm and at which lifetime, plus reads and cold starts; per UTC day, the warm spans and cold starts to the minute (fields). No user ids, no dollars. Kept for the current month only |
| Agent roster | Per salted key pseudonym: last seen, tool, routing, retrieval and hook state, batch reads, Connect version, 24h MCP attempts and reported batch completions (successful, failed, entries returned). No names, paths, or arguments |
| Optimizer health | Hourly per-model outcome counters, hook latency, per-strategy savings, per-strategy shadow horizon counts (groups priced, how many a measured session horizon would have admitted, and that estimate in dollars, never billed), upstream 429/529/4xx/5xx counts, and request counts per retrieval evidence class and reason and per skill exposure and load state, plus how many retrieval counters a full bin dropped (fixed names only) |
| Detailed deployment health | deploymentHealth: two gateway windows, up to 16 optimizer windows, two database probes, up to 32 open or recently closed problems, and fleet counts. Content-free, no user dimension |
| Client tests | clientTests: newest result per plan, up to 20 plans, with receive times. No server result ids or tenants. Visibility only, never billed |
| Heartbeat | Below. Tier standard (default) or floor via PUT /admin/settings |
| Restated limit units | Only after the gateway restates stored savings: for one past month a round, its saving in limit units per billing class, for the month and each day. Counts only, no seats or users |
Detailed deployment health is shared by default, including after an upgrade. It is available to Anyray support. This does not provide notifications or monitoring.
To stop new detailed health data, send {"healthDetailShared":false} to PUT /admin/settings
(content:manage, deployment owner). The choice is audited. The floor heartbeat tier and an
unreadable settings store also stop it. Your local Health page keeps collecting and showing data.
The existing health heartbeat is unchanged. Already shared metrics expire after 7 days;
problem history expires 90 days after closing.
Heartbeat fields
| Field | Contents |
|---|---|
| Health | Versions, uptime, request and error counts (error class names only) |
| Database failures | Allowlisted error code (28P01, ENOTFOUND, 57014), auth mode (password/rds-iam) |
| Upstream rejections | Error enum and field shape (messages.n.content.n.text); an unknown name becomes _ |
| Deployment | Content mode, update method, hostname, DB and volume sizes, reports per Connect version |
| Env vars | Names of set ANYRAY_* vars, never values |
| Optimizer strategies | IDs of the strategies switched on in the optimizer config, never their settings |
| Tool output census | Segment and token counts per output shape and per command family, with trimmed tokens per family; JSON-array item counts in size buckets; request and token totals; the window length and start (fixed names only) |
| Machine state | Memory, CPU, throttles, event-loop delay, ANYRAY_DATA_DIR space, DB pool connections, console queue depth. Sent at both tiers |
ANYRAY_METER_PER_USER=true adds per-user spend and savings keyed by the raw user id. The prompt
cache and warmth blocks are sent with or without it. With the flag on, their pseudonyms can be
linked to those ids, because request counts and savings match. That link reaches the warmth
timeline, to the minute.
Prompt cache by user fields
One entry per user per day. Dollars are rounded to 6 decimal places. A client request is a successful request that billed tokens and reached the provider, so not a keep-warm ping, a failed request or a cache replay.
| Field | Contents |
|---|---|
user | Salted user pseudonym (the seat hash), or * for the users past the top 50 |
requests | Client requests |
readTokens | Cache read tokens |
write5mTokens, write1hTokens | Cache write tokens at the 5-minute and 1-hour lifetime |
writeUnknownTtlTokens | Cache writes on older rows that never recorded the lifetime |
requested1hRequests | Requests whose cache markers asked for 1 hour (all or some) |
loweredRequests, raisedRequests | Requests sent at 5 minutes after asking for 1 hour, and the reverse |
cacheReadSavedUsd | What cache reads saved against fresh input |
cacheWritePremiumUsd | What cache writes cost above fresh input |
keepWarmPings, keepWarmCostUsd | Keep-warm pings sent for the user's sessions, and their cost |
optimizerSavingsUsd | Optimizer savings before subscription scaling, from every successful request: client requests, cache replays and connect hook trims |
mintRewriteUsd | Cache rebuild cost the optimizer's edits forced, over every client request |
gate | Optimizer gate verdicts over every client request: optimizeWins and cacheWins (counts), optimizeEvUsd and cacheEvUsd (expected dollars) |
measuredRequests | Client requests whose cache was measured. Covers only cacheReadSavedUsd, cacheWritePremiumUsd, requested1hRequests, loweredRequests and raisedRequests |
The warmth block has one entry per named user per day, estimated from request rows only. A client request that read or wrote the cache keeps it warm for its lifetime, counted up to the report time. Keep-warm pings are not counted. Where lifetimes overlap, 1 hour counts first. Hourly rows list only hours with activity.
The spans show to the minute when each pseudonym started and stopped using the cache, which approximates working hours. Today's spans are re-sent with every report. A budget alert carries a raw user id and goes out with the next report, so its timing can be matched to these spans.
| Field | Contents |
|---|---|
user | Salted user pseudonym, one of the top 50 above. Never * |
hours[].hour | UTC hour, 0 to 23 |
warm1hMinutes, warm5mMinutes, warmUnknownTtlMinutes | Minutes in the hour the cache was warm at the 1-hour lifetime, the 5-minute lifetime, or an unknown one. A request with no recorded lifetime counts as 1 hour when the provider reported a 1-hour cache write. At most 60 together |
reads, coldStarts | Client requests that read the cache, and ones that wrote it without reading |
minutes.spans | Warm spans of the UTC day as [start, end, lifetime]: whole minutes since 00:00 UTC, end exclusive, lifetime 1h, 5m or unknown. Edges round to the nearest minute |
minutes.coldStarts | Minutes of the UTC day that held a cold start, rounded down, so one can sit a minute before the span it opens |
Vendor connector polling (optional)
Off until a security_admin or owner adds a credential. Metadata comes back and stays local.
| Connector | Host |
|---|---|
| Cursor | api.cursor.com |
| Devin | api.devin.ai |
| GitHub Copilot | api.github.com, copilot-reports.github.com |
A support bundle leaves only when an admin reviews it and clicks Send. Anyray cannot pull it.
Trace export gating
Traces live in your Postgres (anyray_traces, anyray_observations, anyray_trace_session_hours).
The mode gates content before write; optimizer decision sizes are always kept. Hosted MCP connector
credentials (mcp_servers[].authorization_token, an mcp tool's authorization, and every headers
value) are replaced with [redacted] before capture, and connector URLs keep only their origin. This
holds in every mode, large replies included. When a guardrail rewrites a request (a redaction), the
trace holds the rewritten request, whether it was then sent or refused, and the optimizer's
decisions on it keep their sizes only. If a redaction could not finish, or the checks could
not run, and the guardrails then refused the request, the trace holds no request text and no
decision text, only the refusal and its metadata. A streamed reply is stored with the same content
as the reply not streamed, in encrypted and plaintext modes and not at all under off,
including its text, reasoning (redacted thinking included), tool calls and their inputs,
server-tool results (web search results, fetched pages and PDFs, code-execution output, MCP tool
results), generated images (a Gemini image in a chat reply, a Responses image generation result,
each kept up to the 25,000-character field cap) and refusals.
Metadata keeps the upstream URL as origin plus
route-shaped path, and a model id as on the spend row. The trace also keeps promptAnatomy, in its
own column and in every mode (off included): the API dialect, tool and message counts, estimated token counts for
the body, per section and per block kind, cache-marker positions and TTLs, the last 24 messages as
a role plus sizes, and flags for automatic caching, stripped markers and whether the reported usage
describes this body. It holds the conversation's shape, never its text or tool names. A session's traces
together hold the role and size of every message, since each keeps the last 24. Retention: traceRetentionDays
(defaults). Lowering it frees rows, not disk (VACUUM FULL or pg_repack). If
settings are unreachable, that daily sweep is skipped (at most a day lost).
A security_admin or owner can export the bodies of every trace in a time window, from
Export content on Sessions or the API.
A role bundle granting observability:read-content can too, for its own traffic unless it also
holds observability:read. Each row leaves in the mode it was captured under: ciphertext if
encrypted, readable if plaintext, no body if off. An encrypted export plus the content key is
readable content, and a decrypted copy is outside retention, erase and audit. Exports also carry any
connector credentials captured earlier.
Every export response writes trace-content-export audit rows: start before any body leaves,
then complete with the trace count and the users whose traces it carried, or abandoned with a
reason. A start with neither means bodies may have left before the gateway stopped. If the
start row cannot be written, the answer is 503 and nothing is exported.
Connector credentials captured earlier
Gateways released before that redaction stored connector credentials in trace content, mostly on
/v1/responses and /v1/messages. In plaintext every content reader sees them, and in encrypted
anyone with the content key can, so treat every key found there as leaked. Upgrade first so new
traces are captured clean, then run the scrub. It never runs on its own.
As the deployment owner or a security_admin, call it with no body. Nothing changes.
curl -X POST https://<your-anyray-gateway>/admin/v1/traces/connector-credential-scrub \
-H "Authorization: Bearer $ANYRAY_ADMIN_TOKEN"
The report lists the exposed client keys (clientKeys: id, user, owner, type, state), the
third-party values found per connector origin, and how many fields an apply would drop whole
(droppedFields) or keep with an ark_ key replaced (tokenRedactedFields). It never holds a
credential or any trace content.
Send -d '{"apply": true}' with -H "Content-Type: application/json". Each affected row is
rewritten with the values replaced by [redacted], under the current content mode but never a
more exposed one. Under mode off, an affected row's content is removed instead. A row written
since the scrub read it is left for the next run.
A call stops after 20 seconds. While outcome is time_budget, call again with the returned
cursor in the body. Only one apply runs at a time; another gets 409.
The scrub never rotates or revokes anything. Rotate each active key in clientKeys with
POST /admin/v1/keys/{id}/rotate, and have each third-party connector's token replaced.
The size cap cuts a long field to its head and tail, which can leave part of a connector behind. If
the kept text holds a connector, read as JSON keys (an "mcp_servers" list or map, an
"authorization_token" or "server_url" key, a "type": "mcp" tool holding anything the capture
would redact, or the authorization or headers of a tool the cut split), the scrub removes the
whole field, leaves [redacted: connector credentials] in its place, and counts it in
droppedFields. The words headers and authorization do not count, nor do keys with those names
in anything shown not to be a connector (a request's tool call input, say).
What the cut leaves with nothing to say what it was stays: a header without its headers key and
its tool (an X-Goog-Api-Key value, say), a value whose key name the cut left fewer than five
characters of, or a malformed mcp_servers entry holding headers without its mcp_servers key.
Every other field keeps its text. An ark_ key in it (one pasted into a message, say), or what a
cut left of one, is replaced with [redacted] in place, reported in clientKeys when it matches a
key here, and the field is counted in tokenRedactedFields. Rows this deployment cannot decrypt
are counted and skipped.
Durable context stash
Originals moved by reversible strategies such as
observation_mask. Returned only to the same
conversation's provider via anyray_retrieve or
gateway-served retrieval.
| Fact | Value |
|---|---|
| Store | anyray_context_stash, AES-256-GCM in every content mode |
| Guardrails | On chat completions, completions, messages and the Responses API, the input guardrails of the first target a request goes to run before the optimizer. The stash and the semantic cache hold only what they let through from those requests, and a request they refuse stashes nothing. A fallback target's own input guardrails decide what that target gets. Text the first target let through was already allowed to reach a provider, and the stash keeps it under the content mode like any other capture. |
| Guardrails on VS Code, Connect hooks and batch uploads | When an input guardrail can refuse or rewrite a request, the VS Code optimize-only lane and Connect's tool-output hook run it as a seat request would, before the optimizer and the stash. On VS Code, a turn a guardrail refuses or rewrites is not optimized. A Connect output a guardrail refuses is not optimized or stashed, and a redacted one is optimized and stashed redacted. A batch upload's input guardrails run before any line is optimized. One that reads the body and can refuse or rewrite refuses the upload, so its lines are never optimized. |
| Key | ANYRAY_CONTENT_KEY, else derived from ANYRAY_ADMIN_TOKEN |
Placeholder admin token (set-a-strong-local-admin-token), no content key | No new rows; placeholder_key on /admin/health |
| Bad content key | Needs 64 hex chars. Logs ANYRAY_CONTENT_KEY must be 32 bytes (64 hex chars) |
Mode off | Stops new rows; old rows still serve. On Claude Code and Codex seats running a current connect, anyray_retrieve reads trimmed originals from the developer's session file. The optimizer keeps a content-free checksum handle keyed by the tool call ID in its decision pins. It stores no content from these trims. |
| Rotating the admin token, no content key | Orphans old rows (max 48h); they re-create |
| Lifetime | 4 hours, +30 min per retrieval, 48h ceiling. Hourly sweep |
| Erase one person | Also removes rows an agent with delegated attribution stashed for them. A row holding identical bytes the agent also stashed for another of its end users is removed too, and that person's pointer to it stops resolving. |
| Erase during a DB outage | May be missed for up to 90 seconds |
| Erase one tenant at offboarding | Deleting the org in the Billing App erases its stash when the deployment next checks in (within 15 minutes while running). For a deployment that never checks in again, use POST /admin/optimizer/erase-tenant (optimizer:purge). It answers erased: true only after the delete commits. |
The fail-open memo
When the optimizer fails, the gateway replays earlier rewrites from memory so the prompt cache holds. It lives in one process only, holds bytes the provider already got, lasts 1 hour, and is gone on restart. Never on disk or exported.
Files on the developer's machine
anyray-connect writes ~/.anyray/hook-tee/, hook-cache.json, hook-tee-ledger.json,
hook-digest-routers.json, hook-fleet-readback.json (counts your gateway sent back),
hook-arrival-rates.json (rates your gateway sent back for a model id the hook sent it),
hook-arrival-shadow.json and tee-bet-salt (0600 files, 0700 directory). When a Codex
sandbox blocks ~/.anyray, hook-tee copies go to anyray-hook-tee-<uid> in the temp directory
instead, with the same modes. None of these files is uploaded. source-trim/ keeps counts and
sizes only (no commands, paths or output) for 7 days, and only its weekly totals reach your gateway,
as connect health rows. Off switches: ANYRAY_HOOK_TEE=off, ANYRAY_HOOK_CACHE=off.
Retention.
Human surfaces
| Surface | Shows |
|---|---|
/log/stream | Metadata only, in every mode. Bodies and header values read [REDACTED] whatever their type. An injected parameter (override_params) keeps only a number, a flag or a well-formed model id. With ANYRAY_MULTI_TENANT=true, a tenant-bound operator sees only its tenant |
| Console lists | No bodies |
| Trace detail | Metadata. viewer: own and own agents; auditor+: anyone |
| Shared pages | Title and body only to the people a page is shared with. auditor+ lists every page as metadata. Each open by anyone but the author is audited (artifact_open), best-effort: a failed write logs a warning and the page still opens |
Bodies need observability:read-content (security_admin, owner). Each read logs
trace-content-read and each export trace-content-export; if logging fails, bodies are withheld.
Responses carry Cache-Control: no-store.
Admin access control
ANYRAY_ADMIN_TOKEN or console SSO with RBAC. See Security.