N100FN100F
Published on

FinOps for AI

Spend on artificial intelligence arrives by different routes from traditional cloud and grows without anyone seeing the whole of it. How to bring it into the same format, allocate it and measure it per feature.

Service line FinOps Engineering

Spend that is not where you look for it

The cost of artificial intelligence arrives by several routes at once: calls to model APIs, GPU compute in the cloud, AI platforms bought as a service. Each one bills differently and often outside the main cloud account. The result is spend that grows fast and that nobody sees in full.

On top of that there is an underlying difference: in AI, cost depends on user behaviour and product design (how many queries, how long they are, which model is used for each case) far more than on infrastructure.

The same rules, applied to something new

Where AI spend goesA map of AI spend by feature and cost per 1,000 conversations by model size.FINOPS · ARTIFICIAL INTELLIGENCEWhere does AI spend go?MONTHLY AI SPEND BY FEATUREAssistant46 %Search · 27%Summary18 %Other9 %COST PER 1,000 CONVERSATIONS, BY MODELLarge model38 €Medium model9 €Small model2 €If the summary works just as well with the small model,that decision is worth more than any discount.Illustrative data; they do not correspond to any real caseN100F · n100f.com

Everything covered in this series applies, with a few nuances:

  • Normalise: bring the spend from each AI provider into the same format as the rest of the cloud. That way it can be added up, compared and allocated with the same queries.
  • Allocate by feature: the natural owner of the cost is not a server, it is a product function (an assistant, a search, an automatic summary). For that, each call must carry the identity of the feature that originated it.
  • Measure per unit: cost per conversation, per document processed, per active user. It is the only way to know whether an AI feature is profitable.
  • Forecast from usage: AI consumption can multiply with a product change or a spike in adoption. The forecast has to look at usage, not just at past spend.

Levers specific to AI

Beyond the usual levers, AI has its own:

  1. Choose the right model for each task. Not everything needs the most capable and most expensive model.
  2. Reduce what you send and what you ask for: shorter context, more tightly scoped answers.
  3. Reuse results when the same question comes round again.
  4. Set limits per user or per feature, so that anomalous usage does not turn into a surprise bill.

Why now

Many organisations are bringing in AI before they have put their cloud cost in order. Starting with AI spend normalised and allocated from day one is far cheaper than rebuilding it once it is already one of the largest spend lines.

This article closes the series: from understanding that cost is not a bill to building, allocating, measuring, forecasting, optimising and operating it, including for what comes next.

How we apply it: FinOps Engineering.

If your cloud cost keeps growing and nobody knows why, or your architecture depends on whoever built it, let’s talk.

Let’s talk