Prompt caching stores the block of context that repeats between requests —instructions, examples, a base document— and charges you for it once instead of on every request. On workloads with a lot of repeated context, the saving on those tokens reaches 50-90%. It is one of the cheapest levers to implement and it does not touch the quality of the answers.
It is the second of the 7 levers to cut your OpenAI bill and one with the best saving-to-effort ratio.
At a glance
| What it caches | The stable context: system prompt, instructions, examples, base documents |
| Typical saving | 50-90% on the cached tokens |
| Effort | Low: mark the stable block and reuse it |
| When it pays off | Requests that repeat the same long context |
How it works
The first request processes the whole context and stores it in cache for a short window. Following requests that start with the same block do not pay to process that part again: they reuse what is cached and pay only for what is new. The larger and more stable the block you reuse, the more it pays off.
How much it really saves
The saving applies only to the cached tokens, not to the whole bill. If your system prompt and examples take 3,000 tokens and repeat on every request, those 3,000 cost a fraction from the second request onward. At high volume it is the discount you notice fastest without changing anything about the model or the output.
When it pays off (and when it does not)
It pays off when the same context repeats between requests: assistants with a long system prompt, RAG with a fixed base document, classification with many examples. It adds nothing if every request carries a different context, because there is nothing to reuse.
How to turn it on
Separate the stable from the variable: put the block that repeats first (instructions, examples, document) and what changes on each request last. That makes the cacheable prefix as long as possible. Check your provider’s documentation to mark the block; the change usually takes minutes.
Put a number on it
Before touching anything, measure where your spend comes from. Put your context tokens and your volume into the token cost calculator and compare it with what you would pay reusing the stable block: you will see the monthly saving before implementing it. All of this is part of the cost per task that e-ficient measures.
Frequently asked questions
How much does prompt caching save? On the context tokens that repeat between requests, the saving reaches 50-90%. The larger and more stable the block you reuse, the more it pays off.
Does it affect answer quality? No. The model receives exactly the same context; only how the processing of the repeated part is charged changes.
How is it different from RAG? Caching makes the context you already send cheaper; RAG reduces how much context you need to send. They combine well.
At e-ficient we measure the cost per task of companies that already run AI in production and tell you how much you would save with each lever at your real volume. The first audit is free if you start a plan with a three-month commitment and returns a diagnosis within 72 hours.