When the API bill starts to hurt, the first idea is usually the same: run our own model. Sometimes that is the right call, but rarely for the reasons given, and almost always later than people think. The question is not which is cheaper per token, but at what volume the saving covers what it costs to operate.
What gets compared wrong
People compare the API price per million tokens against the hourly price of a GPU, and self-hosting always wins on paper. Half the equation is missing: the GPU is paid for whether it is busy or not, and its real cost depends on utilization. A GPU running at 8% is ten times more expensive per token than the same GPU at 80%.
The costs nobody adds up
| Item | API | Self-hosting |
|---|---|---|
| Compute | Per token consumed | Per reserved hour, used or not |
| Engineering | Almost none | Serving, scaling, updating, watching |
| Availability | The provider’s | Yours, on-call included |
| Model switch | One line of code | Re-evaluate, retune, revalidate |
When it does pay off
There are three situations where self-hosting genuinely wins. The first is high, steady volume: heavy and fairly flat traffic that keeps the GPU busy most of the time. The second is a data constraint that prevents sending data out —regulation, contract, regulated sector— and there the decision is no longer about cost. The third is a very narrow, repetitive task, where a small tuned model does the work of a large one at a fraction of the compute.
When it almost never pays off
Irregular or spiky traffic, a varied catalogue of tasks, or a team with nobody to maintain the infrastructure. If your load has long troughs, you pay for idle GPU. And before moving infrastructure it is worth exhausting the cheap levers: choosing the right model per task, prompt caching and the batch API usually cut more, and in days rather than months.
How to decide with numbers
Work out your real monthly API cost and compare it against the cost of the GPUs you would need for your peak, not your average. Add engineering time at market rate. If the saving does not comfortably exceed that total, it does not pay off. Put figures on it with the token cost calculator and always measure in cost per task, the only thing comparable across both options.
Frequently asked questions
At what volume does self-hosting pay off? There is no universal figure, because it depends on utilization, not volume. The signal is having enough steady traffic to keep the GPU busy most of the time; with long troughs, the API is cheaper.
Is an open model cheaper than a paid one? Per token of compute, yes. Adding engineering, availability and idle GPU, not always. The honest comparison is cost per task, all in.
Can I mix the two? That is the most common setup in practice: an own model for the high-volume task and APIs for the rest. Splitting by task usually beats picking a single provider.
At e-ficient we measure the cost per task of companies already running AI in production and tell you how much you would save with each lever at your real volume. The first audit is free if you start a plan with a three-month commitment and returns a diagnosis within 72 hours.