GPU spend can go from a small line item to a runway problem faster than most founders expect. One product release, one enterprise customer, or one model change can turn a tidy cloud bill into a monthly surprise.
That is why GPU forecasting is not only a finance exercise. It is a growth decision, a pricing decision, and a cash decision. If you get it wrong, you can sell a product that gets more expensive every time customers love it.
The fix is not fancy. Start with how your product is used, separate training from inference and support work, then turn that usage into a monthly model you can trust.
Why GPU spend needs its own forecast
Most software costs are fairly well-behaved. Your CRM, payroll system, and core SaaS tools tend to move in steps. GPU usage does not. It can jump overnight, because it follows product behaviour.
A flat monthly average hides the real story. If a customer rolls out your AI feature across a whole team, inference can spike within days. If your engineers re-run training after a data issue, the bill can move again. If latency slips and you add more capacity, spend rises before revenue catches up.
That is why GPU spend should sit in its own forecast tab, not buried in “cloud” or “infrastructure”. If you fold it into overhead, you lose sight of the one cost that can scale hardest with product success.
The three cost drivers that move fastest
For most AI startups, three workloads matter most: training, fine-tuning, and inference.
Training is heavy, lumpy, and irregular. You might run it once a quarter, or several times in a month if the model is still moving. Fine-tuning is usually smaller, but it can still create sudden bursts of cost. Inference is different again. It is tied to live usage, and it often becomes the larger ongoing bill.
Each one scales in its own way. Training depends on dataset size, model choice, and experiment cycles. Fine-tuning depends on how often you adapt the model. Inference depends on users, prompts, tokens, latency targets, and uptime.
If all three sit in one bucket called “AI infra”, you are guessing, not forecasting.
Why early guesses are often too low
Founders usually underestimate GPU spend for simple reasons. They assume usage will grow in a straight line. It rarely does.
They also forget the waste around the edges. Retries, failed runs, bad prompts, test traffic, evaluation jobs, and over-provisioned capacity all add cost. None of those look dramatic on their own. Together, they matter.
Then there is product growth. A feature that feels cheap at 100 users can look completely different at 2,000 active users. More prompts, longer responses, and heavier workflows multiply compute quickly. This is why GPU spend needs proper modelling, not a quick average from last month’s invoice.
Build your forecast from the product, not from the cloud bill
The best GPU forecast starts with product activity. Last month’s bill is useful, but it is only a rear-view mirror. A founder needs a forward view.
Work backwards from what customers will actually do. How many active users do you expect? How often will they hit the model? How many tokens, images, or requests sit inside each action? Are you serving live responses, batch jobs, or both?
That gives you a repeatable model. It is also much easier to defend in a board pack or fundraise. If you are building this into a wider plan, it should sit inside an investor-grade SaaS financial model, not a loose spreadsheet with one cloud number at the bottom.
Map user actions to GPU demand
Start with one user journey. Keep it boring and clear.
Say 2,000 users are active this month. Each sends 20 prompts. That is 40,000 prompts. If each prompt triggers 1.2 model calls, because some requests use moderation or retries, you are already at 48,000 calls. If average output length rises after a new feature, cost rises too.
The same logic works for image generation, transcription, search, or agents. Pick the product action, then map the compute behind it. Once you do that, finance and product are looking at the same thing.
Separate training, inference, and support workloads
Do not lump everything together. Training runs, live inference, development environments, monitoring, evaluation, and QA all behave differently.
Live inference scales with customers. Training may depend more on roadmap timing. Evaluation can rise after each release. Dev and test work might stay fairly steady, even if customer usage is flat.
When you mix them, you lose the driver behind the spend. When you split them, you can ask better questions. Did cost rise because traffic jumped, because model quality work increased, or because the team ran too many experiments?
Choose the right unit for each workload
Use the unit that matches how the cost is created. Training is usually easiest in GPU-hours. Inference may be clearer in requests, images, or tokens, then translated into GPU usage or provider charges. Support work might fit jobs, runs, storage volume, or monitoring events.
Do not force everything into one metric. A clean model is better than a clever one.
There is also a practical point here. If your provider bills in one unit and your product team measures another, bridge the two with a simple assumption table. That keeps the model readable, while still matching the bill.
Turn usage into a monthly cost model
Once you have workload volumes, the next job is simple: attach a rate. That means cost per GPU-hour, cost per request, cost per million tokens, or a blended monthly cost for reserved capacity.
As of 2026, on-demand rates still vary a lot by provider. Entry-level GPUs used for lighter inference, such as L4 or A10-class machines, can land around $0.50 to $2.50 per hour. A100-class capacity is often nearer $2 to $5 per hour. H100 is materially higher. Specialist providers may beat hyperscalers on price, while reserved or spot capacity can cut rates further, with trade-offs on flexibility.
Estimate the cost per GPU-hour or per token
Every forecast needs a real rate card. Use the machine type you actually plan to run, not the cheapest number you saw on social media.
A lower hourly rate does not always mean a lower bill. If the model is poorly matched to the GPU, or utilisation is weak, your effective cost per request gets worse. The right question is not “what is the cheapest GPU?” It is “what gives us the lowest cost for the workload we have?”
Using illustrative rates, a simple monthly model might look like this:
| Workload | Monthly volume | Example rate | Example cost |
|---|---|---|---|
| Training | 320 GPU-hours | £2.40 per GPU-hour | £768 |
| Inference | 720 GPU-hours | £0.95 per GPU-hour | £684 |
| Support and evaluation | fixed monthly estimate | £218 | £218 |
| Total | £1,670 |
The point is not the rate. The point is the structure. Once the layout is right, updating the numbers becomes easy.
Add in the costs people forget
GPU time is only part of the bill. Storage, data transfer, orchestration tools, monitoring, testing, failed jobs, and idle instances all belong in the model.
At first, these extras can look small enough to ignore. That is a mistake. If your GPU line is growing quickly, the surrounding costs usually follow.
Keep them visible. Add a separate section for support costs rather than hiding them inside a blended compute rate. You will spot waste faster that way.
Test the model against real invoices
A forecast only gets good when you compare it to actual spend. Do that monthly.
Take the live invoice and line it up against your model. Where did the gap come from? Higher traffic, lower utilisation, more retries, new workloads, or bad assumptions on token volume? Then update the drivers, not only the final number.
This is also where currency matters for a UK startup. Many providers bill in US dollars. If your accounts and runway are in pounds, add a simple FX assumption so the finance model matches reality.
Use scenarios to protect runway and plan growth
One forecast is not enough. You need at least three: low, base, and high.
Why? Because AI products do not grow neatly. A new launch can double inference. A big customer can change your traffic pattern in a week. A model switch can increase cost before pricing catches up. Scenario planning gives you room to think before the cash does the talking.
It also helps with hiring, pricing, and fundraising. GPU spend is part of burn, gross margin, and runway, right alongside the key SaaS metrics for growth tracking.
Build a base case your team can trust
Your base case should be boring. That is a good thing.
Use current product usage, committed customer launches, known roadmap changes, and a sensible growth rate. Do not build the middle case from wishful thinking. Build it from what the team can explain in one meeting without hand-waving.
This becomes the working version for monthly planning.
Stress test the upside and downside
The high case should model faster customer growth, heavier usage per account, model changes, or stricter latency targets that need more capacity. The downside case should assume slower adoption, while keeping fixed costs honest.
That second point matters. Even if usage is light, some costs remain. Reserved capacity, minimum contracts, and support tooling do not disappear because the sales cycle slipped.
A good scenario model shows where your runway breaks and where your unit economics improve.
Set simple triggers for when to update the forecast
Do not leave the model untouched for a quarter. Set clear triggers.
Update it when a large customer goes live. Update it when you ship a feature that changes prompt length or request frequency. Update it when you switch models, see a rise in retries, or spot churn that changes usage assumptions.
Keep it simple. If reality moves, the forecast moves with it.
Build the model before the bill arrives
GPU spend is forecastable. It only feels unpredictable when training, inference, and support work are mixed together, and when finance starts with the invoice instead of the product.
Founders get better control when they use simple units, real provider rates, and monthly checks against actual spend. That is how you protect runway without slowing growth.
Build the model now, while the numbers are still small enough to fix.
Over 12 years across Big Four audit, investment banking and corporate advisory. Kishen works with UK SaaS and AI companies on financial strategy, fundraising and board-level CFO support. ICAEW regulated. Big Four trained. Based in London.