Tensormesh

Tensormesh serves open-weight models on a KV-cached vLLM fleet and bills cached input at $0. Anyray routes to it as a first-class provider and prices the cached share at zero.

Property	Value
Provider id	tensormesh
Gateway speaks	OpenAI /v1/chat/completions; Anthropic /v1/messages is translated
Credential	A Tensormesh inference API key (Deploy, then Serverless)
Base URL	https://serverless.tensormesh.ai/v1, built in
Model ids	Hub names as served, such as Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8

Set it up

1. Create an inference API key in Tensormesh under Deploy, then Serverless.
2. Save it on the Providers page as the tensormesh key, or set ANYRAY_PROVIDER_KEY_TENSORMESH (provider keys).
3. Route a model to it: on the Routing page pick provider tensormesh and a served model id, or send "provider": "tensormesh" in x-anyray-config (routing).

The served catalog is GET https://serverless.tensormesh.ai/v1/models, no key needed.

What Anyray meters

Tensormesh reports cached input as prompt_tokens_details.cached_tokens. The gateway prices that share at $0 for every model whose price card lists cached tokens at $0, and at the list input rate for the rest. Streams ask Tensormesh for the final usage frame, so a streamed turn records the same split as a blocking one.

A model on the served list but not yet in Anyray's price table prices at the provider floor, the cheapest Tensormesh rate, never at the house rate.

Claude stays on Anthropic

Tensormesh serves open-weight models only. A Claude Code seat can be routed here for a Qwen or Kimi coding model; Claude itself keeps Anthropic's own prompt cache.