A token limit is the ceiling on how much text an AI model can handle in one go, counted in tokens rather than words or characters. It covers what you put in and what the model sends back, all within the same budget.
A handy way to picture it is a whiteboard. You can write a lot on it, but once it is full you have to erase something before you can add more. The token limit is the size of that whiteboard. If a conversation or document is too big to fit, the model cannot see all of it at once, and the part that does not fit is left off the board. This limit is closely tied to the model’s context window, which describes the same space from the other direction.
The limit matters in practice because real work often involves long contracts, transcripts or chat histories that exceed it. The usual fix is to split the text into smaller tokens and feed only the relevant pieces, or to summarise older parts so they take up less room.
There is also a subtle trap with long contexts. Even when a document technically fits, a model tends to pay most attention to the start and the end and gets hazy in the middle, so cramming a file right up to the limit often gives worse answers than feeding a tight, relevant slice. Bigger is not always better.
Cost moves with the same dial. Since you pay per token, sending the full 100,000-token limit on every question is far pricier than a focused 2,000-token prompt that lands the same answer.
At TopDevs we design AI features around the token limit from the start, chunking documents and pruning context so a long file never quietly gets cut off mid-task.
