Skip to main content
An agent resends its whole prompt on every turn: its instructions, its tool definitions and the conversation so far. Providers can serve a repeated prefix from a cache, and cached input costs about a tenth of the normal price. Cruq AI sets this up for you; there is nothing to configure.

​
Which models

  • Claude and Gemini through OpenRouter only cache up to a marker in the request, so Cruq AI adds two. One goes after the agent’s own instructions, so a second run of the same agent within five minutes starts warm. The other goes after the latest message, so each turn of a tool loop pays full price only for what is new. Run context and the plan come after the first marker, because they change from run to run.
  • OpenAI models cache a long prefix by themselves.
Writing to the cache costs a little more than a normal read (1.25x on Claude), so the second marker is only added for agents with tools, where there is a next turn to read it. Providers also skip caching below a minimum length, about 1,000 tokens on most Claude models. On a six-turn Claude Sonnet run with a calculator tool, this cut the run’s cost from $0.063 to $0.024, and to $0.016 when the agent had run in the previous five minutes.

​
Seeing it work

Each model call in a run’s trace shows how much of its input came from the cache, for example 3.1k in (97% cached). Over the API it is cached_input_tokens and cache_write_tokens on the llm_call step.

​
Keep the instructions cacheable

The first marker only pays off if the agent’s instructions are the same on every run. Keep anything that changes between runs, like today’s date or a customer’s name, out of the instructions and pass it as run context instead. Run context goes after the marker, so it never breaks the cache.