Cost per token is the price an AI provider charges for each small piece of text a language model processes. A token is roughly three-quarters of a word, so a typical paragraph is a few hundred tokens. You pay for what goes in and what comes out, and those amounts are multiplied by a per-token rate to produce your bill.
Think of it like a taxi meter that ticks for every word, not every mile. A quick one-line question barely moves the meter. But paste in a 40-page contract and ask for a full rewrite, and the meter runs hard, because you are charged for the whole document going in and the whole rewrite coming out. This is why a large context window is powerful but not free: every extra token you feed the model is billed.
Prices differ sharply between models, and output tokens almost always cost more than input tokens. A small, fast model might be cheap enough to run on every request, while a top-tier model is reserved for the hard questions. That trade-off is the heart of model routing.
The numbers feel tiny until traffic arrives. A single answer might cost a fraction of a cent. But run that feature ten thousand times a day, and the fractions stack into a real line item. This is where small design choices pay off. Trimming a bloated system prompt, summarising a long history instead of resending it, or caching the parts that never change can all cut the count before a request even reaches the model. Watch the per-feature spend the way you watch server bills, not as a fixed fee but as something that scales with every user you add.
At TopDevs we design AI features with the meter in mind, trimming wasted context and matching each task to the right model so the bill stays predictable as usage grows.
