How much does an AI feature cost: tokens, infrastructure and development

The cost of an AI feature has three parts: development (integration, data, quality checks), ongoing model calls billed by tokens, and the infrastructure around them — retrieval, storage, monitoring. Development is usually the biggest one-off item, while token costs grow with your user base, so they need to be estimated before launch.

Here is how to calculate each part, where to save and which costs are most often forgotten.

What makes up the cost of an AI feature?

The cost of an AI feature consists of one-off development costs and ongoing running costs: model calls, infrastructure and quality upkeep. The model call is only part of the spend, and often not the largest.

Item	Cost type	What it includes
Research and prototype	One-off	Checking whether the model solves the task on your data
Development	One-off	Context retrieval, prompts, validation, UI, integrations
Evaluation set	One-off and ongoing	Examples with correct answers, run on every change
Model calls	Ongoing	Paying for input and output tokens
Infrastructure	Ongoing	Vector search, queues, log storage, GPUs for self-hosted models
Support	Ongoing	Error review, prompt and model updates, cost monitoring

We described the architecture these items come from in how to integrate an LLM into your product.

How do you calculate token costs?

Token costs follow a formula: number of calls × (input tokens × input price + output tokens × output price). Providers publish their prices, which differ between models by an order of magnitude or more, so run the calculation for several options.

Illustrative example: a support assistant on a knowledge base
  requests per month               20,000
  input tokens per call            3,000  (question + retrieved passages + instructions)
  output tokens per call           400
  input price                      X per 1M tokens
  output price                     Y per 1M tokens

Monthly cost = 20,000 × (3,000 × X + 400 × Y) / 1,000,000
             = 60 × X + 8 × Y

Plug in current prices for the models you are considering and you will see how model choice drives the budget. Note that input tokens usually far outnumber output tokens: document context and instructions make up most of the request.

Photo: Unsplash

What quietly inflates costs

• a long chat history sent in full with every message;
• excess context — ten document passages where three would do;
• repeated calls after validation failures;
• agentic flows where the model makes several calls in a row for one user request.

How do you reduce model costs?

Reduce model costs by matching the model to each task, trimming context and using provider features for repeated requests. Savings of several times are often possible without losing quality.

What works:

1. Routing by difficulty: simple requests go to a fast, cheap model; hard ones to a powerful model.
2. Caching: identical answers from your own cache; repeated context through prompt caching where the provider supports it.
3. Batch processing: many providers process non-urgent background jobs more cheaply in batch mode.
4. Shorter context: fewer but more precise retrieved passages; compressed chat history.
5. Limits: maximum response length and per-user, per-day caps.

How much does developing an AI feature cost?

Developing an AI feature costs roughly as much as a regular feature of similar scope, plus work on data and quality evaluation. A prototype on an off-the-shelf API takes days; a reliable customer-facing feature usually takes several weeks, with most of the time spent on data preparation and quality.

What drives the scope:

• how ready and clean the data behind the feature is;
• whether document retrieval is needed, and over how much content;
• how costly mistakes are and whether results need human review;
• how many integrations: CRM, knowledge base, ticketing system;
• data requirements: whether cloud providers are allowed or self-hosted models are required.

To check that a quote for this work is complete, see how to read a software development quote.

When is a self-hosted model cheaper than a cloud API?

A cloud API is cheaper at the start and at moderate volume: no GPUs or operations team, and you pay only for actual calls. A self-hosted open model can win at large, steady volumes or when data cannot be sent to an external provider.

What to weigh:

• GPU servers cost money around the clock, even with no requests;
• you need people to update, scale and monitor the models;
• open models' quality on your task must be verified on your own example set;
• a hybrid — cloud for hard tasks, a self-hosted model for high-volume simple ones — is often optimal.

In short

• An AI feature's cost is development, tokens, infrastructure and quality upkeep.
• Estimate tokens with the formula and current prices for several models; input tokens usually dominate.
• Routing by difficulty, caching, batch processing and short context save the most.
• A reliable feature takes weeks to build, mostly spent on data and quality evaluation.
• Cloud APIs win at the start; self-hosting pays off at large steady volume or with strict data requirements.