A token is the basic unit of text that a language model works with. Before a model can read your message it chops the text into tokens, and when it answers it produces one token at a time. A token is often a whole word, but longer or unusual words get broken into smaller pieces.
Think of tokens like Lego bricks for language. The word ‘cat’ might be a single brick, while ‘unbelievable’ could be three bricks snapped together: ‘un’, ‘believ’, ‘able’. The model never sees letters or full sentences the way you do; it only sees the stream of bricks. The step that does this splitting is called tokenization, and how many bricks fit in one conversation is set by the model’s context window.
Tokens are also the unit of price. Almost every AI provider bills per token for both the text you send and the text you get back, so a long prompt with a long answer costs more than a short exchange. Watching token counts is how teams keep AI features affordable at scale.
The split is not the same in every language. English is cheap because the tokenizer was trained mostly on it, so common words map to single tokens. Dutch, German or Japanese often break the same idea into more pieces, which means the same sentence can cost noticeably more. And emojis, code and odd symbols can each eat several tokens on their own.
This is why a token is the unit that matters in practice, not the word. When you design a prompt, the real budget you are spending is measured in tokens, not characters or sentences.
At TopDevs we keep an eye on token usage when we build AI features, trimming prompts and reusing context so a client pays for results rather than wasted words.
