GPT-5.6 prompt cache: 0.66% hit rate in production, 99.96% in a probe
How a changing tail defeats exact-prefix caching, and why our production fix reached only 9.86%.
One long agent session on GPT-5.6 Luna, served through OpenRouter, wrote almost its whole prompt to the cache on every round. This issue gives the numbers and the request shape that caused it.
The short answer
- Production hit rate was 0.66%. In 39 rounds with prompts of 401K to 515K tokens, cache writes were 99.34% of reported prompt tokens.
- A strict extension of the prompt read 99.96% from the cache in a controlled probe: 45,786 of 45,805 cached tokens, against 1,215 for the broken shape.
- The fix is not complete. After deployment the token-weighted hit rate was 9.86%, because every new turn still read zero cached tokens.
Production before and after
| Metric | Before, August 22 | After, August 24 |
|---|---|---|
| Model rounds | 39 | 20 |
| Token-weighted cache hit rate | 0.66% | 9.86% |
| Cache writes / prompt tokens | 99.34% | 40.66% |
| Model cost per round | $0.243 | $0.188 |
The two traces are not a like-for-like comparison: the second has 20 rounds and prompts of 441K to 469K tokens.
When it applies
- GPT-5.6 and later: the default breakpoint is the latest user or tool message and does not search backward. A timestamp or regenerated runtime block in that message makes the model write the changing prefix again.
- The request shape to use: put stable instructions, tools, schemas and durable history first. Put timestamps, request IDs and retrieved state after an explicit breakpoint.
- What is still open: the explicit breakpoint helps within a multi-call turn, but the reusable boundary is lost when the next turn starts. The cached-token comparison (37.7 times) is not a claim that every bill falls by 37.7 times.
The full post has the request examples, how to choose cache keys, the metrics to track and a five-request debugging probe.