Data boundary
Where content travels, and what leaves your environment: metadata only.
- Billing, connectors, spend rows, and logs contain metadata only.
- Trace content follows the deployment's content mode.
How the boundary is enforced
Seven independent layers, so one misconfiguration does not expose content:
| Layer | Guards against | Control |
|---|---|---|
| Encryption at rest | Content readable on disk | ANYRAY_CONTENT_KEY |
| Content mode & fail-safe | Content being kept at all, or silently escalating | the content mode, ANYRAY_ALLOW_PLAINTEXT |
| Content-free spend store | Content leaking into governance data | Metadata-only by design |
| End-point evidence | Device inventory or arbitrary evidence reaching Anyray | Fixed metadata schema and explicit deletion |
| Trace export gating | Content leaving in exported traces | the content mode |
| Human surfaces | Humans seeing content in the UI | the content mode + RBAC |
| Admin access control | Unauthorized access to data and settings | ANYRAY_ADMIN_TOKEN (+ optional SSO / RBAC) |
Encryption at rest
In encrypted mode, which is the default, content is encrypted with AES-256-GCM before it
touches disk, using ANYRAY_CONTENT_KEY (32 hex bytes, openssl rand -hex 32). It fails closed:
there is no plaintext fallback, and the gateway refuses to boot without a valid key. Decryption is an
offline, authorized step, and the console never decrypts content for display.
Content mode & fail-safe
One org-wide content mode decides how much content is kept. Set it in the console, from the
account menu under Content privacy, or with PUT /admin/settings. It lives in the shared database, so it applies to every replica, and
every change is audit-logged.
| Mode | Behavior |
|---|---|
encrypted (default) | Content stored as ciphertext (AES-256-GCM via ANYRAY_CONTENT_KEY). |
off | No content stored, so nothing to decrypt. |
plaintext | Raw content, readable by admins. Requires ANYRAY_ALLOW_PLAINTEXT=true. Debugging only. |
The mode only ever degrades toward off, and never escalates. Without the gate below, a
plaintext setting falls back to encrypted when a key is set, and to off otherwise.
ANYRAY_ALLOW_PLAINTEXT=true
Debugging in plaintext, and the gateway-less paths
Add the gate, restart, then select plaintext on the Privacy page. Remove the gate when you are
done. ANYRAY_CONTENT_MODE governs only the gateway-less paths, meaning BYO /v1/record and attach
mode.
Content-free spend store
The gateway writes one metadata row per request. No field carries content, not even ciphertext.
A request routed through a custom provider host is flagged with a boolean (via_custom_route), and
the hostname is never stored. Rows are kept per ANYRAY_SPEND_RETENTION_DAYS, and unset means kept
for good.
| The row holds | Detail |
|---|---|
| Who ran it | User and team, from x-anyray-metadata |
| Which tool | A fixed id such as claude-code or codex, never the raw User-Agent |
| What ran | Model and provider |
| Which route | The request path alone, such as /v1/messages. The query string is dropped before the row is written, a seat credential in the path is masked, and a segment that could not belong to a route is replaced with :invalid. |
| The numbers | Token counts with the prompt-cache split, cost, savings, latency, status |
Two more tables live in the same Postgres and hold no content either.
| Table | What it holds |
|---|---|
anyray_auth_events | Reason-coded auth and cap rejections for the admin Diagnose view. Kept 30 days, never sent to Anyray. |
anyray_spend_connector_events | Vendor metadata from configured spend connectors. Never affects anyray_spend, token caps, request governance, or Billing rollups. Expires after 90 days, or sooner when ANYRAY_SPEND_RETENTION_DAYS is shorter. Owner-level GDPR endpoints cover it, and content fields are never imported. |
Where end-point evidence lives
The endpoint-control service runs beside the gateway and keeps end-point compliance evidence in your own Postgres. None of it crosses to Anyray. The metering rollup carries only content-free fleet counts: reachability, host and compliance counts, failing-policy counts, and the policy-pack version. The built-in agent and the customer-managed connectors (Jamf, Intune, Kandji, Fleet, EDR, custom) feed the same service and the same bounded fields.
| Field | Treatment |
|---|---|
| Raw device identifiers | Hashed at the store boundary |
| Credentials | Stored only as SHA-256 hashes |
| Inventory metadata, query rows, logs, device tokens, script output | Discarded, never persisted |
Rows stay until an operator deletes the host or the organization. Full field inventory and the deletion path: Endpoints.
What the Billing app rollup sends
Every deployment sends a content-free Billing rollup every 15 minutes, and after health changes. It never carries prompt or response content.
| Part | What crosses |
|---|---|
| Seats | A distinct active-seat count, opaque seat hashes (salted pseudonyms, the salt never leaving your deployment), and a distinct attributed-user count covering every billing mode. A single number, never a list of identities. |
| Usage totals | Requests, tokens, cost, and month-to-date savings, bucketed by model and provider. |
| Deployment heartbeat | Versions, uptime, per-leg health, request and error counts (error class names only) both gateway-wide and for inference routes alone, the class an upstream gave for a rejected request (its own error enum, plus the SHAPE of the request field it objected to, such as messages.n.content.n.text, where any field name we do not recognise as part of the request schema becomes _ — the upstream's message text is never sent, and no value from your request can appear), content mode, update method, auto-soft-update state, replica hostname, the spend and trace database's size in bytes and, when the deployment can know it, its volume's capacity and free bytes, a count of health reports per anyray-connect version in use, and the names of set ANYRAY_* env vars. Values never cross. |
| Optimizer health | Hour-by-hour counters per model: optimize outcomes (applied, skipped, timeout, error, and normalized reason tokens), hook latency percentiles, per-strategy fired counts and tokens saved, and upstream provider failure counts (429, 529, 4xx, 5xx). Counters and enum tokens only, with no per-request data and no user field of any kind. |
This powers Anyray-side fleet support ("your deployment has been unhealthy since 09:12") with no
access to your environment. Content privacy → Shared shows the literal last payload that
crossed. The heartbeat tier is standard by default, or floor for the mandatory core; set it with
PUT /admin/settings.
ANYRAY_METER_PER_USER=true adds a per-user spend and savings breakdown, still counts and dollars
only. Leave it unset to keep the rollup fully pseudonymized.
Optional vendor connector polling
Connectors remain off until an admin adds a credential. In single-tenant mode, security_admin
or owner may configure them. In fleet mode, only the deployment owner may. The gateway then
polls these hosts:
| Connector | Outbound HTTPS host |
|---|---|
| Cursor | api.cursor.com |
| Devin | api.devin.ai |
| GitHub Copilot | api.github.com, then GitHub-issued report downloads pinned to copilot-reports.github.com |
Calls send the credential and report window, and responses contain metadata stored locally. No inference content is sent. Remove the credential to stop polling.
Support bundle (customer-initiated)
A separate, optional egress for escalation. An admin generates the bundle, reviews the redacted JSON on screen, then separately clicks Send. The gateway regenerates it at send time, so the server decides what leaves, not the browser. The egress is audit-logged and can never be triggered or pulled by Anyray.
Trace export gating
Traces persist to Postgres inside your environment (anyray_traces and anyray_observations).
The mode gates content before it is written: ciphertext in encrypted, omitted in off, and raw
only in deploy-gated plaintext. Tokens, cost, provider, and attribution are always plain metadata.
Set the trace window on the console Privacy page, or with PUT /admin/settings and
traceRetentionDays, from 1 to 3650 days. It is separate from spend retention, because the trace
tables hold content and grow far faster. Unset, it follows ANYRAY_SPEND_RETENTION_DAYS, else 90 days.
| Situation | What happens |
|---|---|
| You lower the window | Rows are deleted, but Postgres keeps the disk and reuses it for new writes. Usage plateaus rather than shrinking. VACUUM FULL or pg_repack returns it to the OS. |
| The settings store is unreachable at sweep time | That sweep prunes nothing and logs why, instead of falling back to a window no operator chose. Deletion is irreversible and the sweep is daily, so you lose at most a day. |
Durable context stash
output_externalize can move a bulky tool output
off the request and serve it back on demand.
observation_mask stashes masked observations the
same way.
Originals are held in memory by default. The opt-in durable tier persists them in your spend
Postgres (anyray_context_stash), gated like the trace store: an AES-256-GCM blob in encrypted,
nothing new written in off. Switching to off stops new rows but still serves spans stashed
beforehand, so a trim already in flight can be fetched back instead of lost mid-conversation.
Existing rows go on their TTL, or immediately if you purge. The handle is a content-free keyed HMAC,
recall is scoped to the stashing user, and expired rows are swept hourly.
| Lifetime | Value |
|---|---|
| A stashed original expires | 4 hours after it was stashed |
| Each successful retrieval extends it by | 30 minutes |
| Ceiling on those extensions | 48 hours |
output_externalize default | 24 hours, and it has no in-memory mode |
Why retrieval extends the clock
The read half is what keeps a trim reversible. An elision marker in a conversation is permanent, so a marker the model uses stays resolvable, while one nothing reads expires on its original schedule.
The ceiling bounds how far retrievals can carry one original, not how long recurring content stays stored. A handle is a keyed hash of the content, so output the agent produces again is stored again and starts a fresh 4 hours. Nothing is kept without being re-produced or re-read.
The fail-open memo
The optimizer is best-effort, so the gateway forwards the request without it on a timeout or an error. A turn that skipped it must still send the provider the same bytes the turns around it sent, or the prompt cache is rewritten from the first changed byte. So the gateway keeps a small per-session memo of what the optimizer turned each tool description, system block, and message into, and replays those substitutions on the turn it skips.
That memo holds message content, so it belongs on this page. What bounds it:
| Where it lives | One gateway process's memory. Never written to disk or to Postgres, never exported, never sent to Anyray |
| What it holds | Only bytes the provider has already received, as the optimizer rewrote them |
| Scope | One session. A replay can only ever restore that session's own rewrites |
| Retention | The provider cache horizon: 1 hour. A rewrite that points at a stashed original is not kept at all |
| On restart | Gone. The next skipped turn sends the original bytes, which costs one rewrite |
It is dropped early whenever the rewrite stops being right: an optimized turn that hands a block back unchanged drops that block, and a provider rejecting the rewritten body, or a paused subscription, drops the whole session. A GDPR delete covers the stores that persist; nothing here outlives the hour or the process, so there is nothing for it to reach.
Files on the developer's machine
anyray-connect trims oversized tool output on the developer's own machine. Two paths keep bytes
locally under ~/.anyray/. Neither is ever uploaded, and no spend row, trace, or Billing rollup is
derived from them.
| Store | What it holds | Retention |
|---|---|---|
hook-tee/ | The untrimmed text of a tool output the hook trimmed, so the full output: Read … footer the agent was given still resolves | 30 days after the last footer named the file, 90 days from the first write, and 128 MiB in total, whichever binds first |
hook-cache.json | The optimization verdict for an output already seen, so an identical one skips a second round trip | 24 hours, 200 entries |
Both are written mode 0600 inside a mode 0700 directory, and hold nothing the agent's own
transcript does not already hold. The content mode does not reach them, because a stored output
becomes a gateway concern only when the agent reads it back into a prompt. ANYRAY_HOOK_TEE=off and
ANYRAY_HOOK_CACHE=off stop the stores writing.
Human surfaces
| Surface | What a human sees |
|---|---|
The live /log/stream view | [REDACTED] for content fields in every mode. Request logs stay metadata-only even when trace capture is deploy-gated plaintext. |
| Console list views | No request or response bodies at all. |
| Trace detail | Open to any role with observability:read (viewer and above). Shows model, tokens, latency, cost, and optimizer decisions. |
Seeing the bodies inside a trace needs observability:read-content, held by security_admin and
owner only. Every other role gets a redacted response, not a 403. What they see still follows the
mode: ciphertext in encrypted, nothing in off, and readable only in plaintext.
Admin access control
The console and every /admin/* endpoint sit behind admin auth: the break-glass
ANYRAY_ADMIN_TOKEN, compared in constant time, and optionally per-operator console SSO with
RBAC. See Security → Console access and RBAC.