Tokens, Cost and Length Limits

Models do not read characters or words. They read tokens — fragments produced by splitting text according to a fixed vocabulary. Almost every counterintuitive thing about limits and pricing comes from this.

What a token is

A common English word is usually one token. A longer or unusual word splits into several. Punctuation, spaces and line breaks are tokens.

A rough rule for English prose: about four characters per token, or roughly three-quarters of a word. A thousand words is around 1,300 tokens.

Code tokenises less efficiently because of punctuation and identifiers.

Non-English text is frequently much worse. Languages that were less represented in the tokeniser's training split into more, smaller pieces. The same passage in some languages costs several times more tokens than in English — and consumes proportionally more of the context window.

This is a real and underdiscussed inequality: users writing in some languages pay more and hit limits sooner for identical work.

What is counted

Input tokens: your message, the conversation history, attached documents, system instructions, retrieved material.

Output tokens: what the model generates.

Output is typically several times more expensive per token than input, because generating requires a pass through the model per token while input is processed in parallel.

Everything is re-counted each turn. A fifty-message conversation sends all fifty messages with message fifty-one.

Why costs surprise people

Long conversations scale badly. Cost per exchange rises as history accumulates. Ten short conversations cost much less than one long one covering the same ground.

Attached documents are charged every turn they remain in context, not once.

Reasoning modes generate large amounts of intermediate text that you may not see and may still pay for. Check whether a product bills for reasoning tokens.

Retrieval adds input. Every search result inserted is input you are charged for.

Asking for brevity saves real money where you pay per use, because output is the expensive side.

Limits you actually meet

Context limit: total input plus output. Exceed it and material is dropped or the request fails.

Output limit: the maximum a single response can be, usually much smaller than the context window. This is why long documents come back truncated.

Rate limits: requests or tokens per minute.

Plan limits: messages per period on consumer subscriptions, frequently expressed in messages rather than tokens, which makes them hard to predict.

Practical consequences

Ask for the length you want. "In about 200 words" is cheaper and usually better than an unbounded answer.

Do not paste an entire document when a section will do.

Start new conversations when the topic changes.

For repeated work, prefer many short calls to one long thread.

Watch for prompt caching, which several providers offer. Where a large unchanging prefix — a document, a long instruction set — is reused, cached input is billed at a fraction of the normal rate. For repeated queries against the same material this is a large saving and it requires structuring requests so the constant part comes first.

Estimate before committing to volume. Multiply expected tokens by the published rates for a realistic figure. Doing this before building a workflow around a paid model avoids an unpleasant first invoice.

Why the count matters beyond money

Quality degrades near the limit. A model approaching its context ceiling handles the material less reliably, and truncation happens without announcement.

Long inputs dilute attention. Material in the middle of a very long context is used less reliably than material at the edges.

Output limits produce silent truncation. A response cut off at the limit may end mid-sentence, or may end plausibly while omitting what you asked for. Asking for a structured format makes omissions visible.

Counting them

Every major provider publishes a tokeniser or a counting tool. If you are building anything where cost matters, count a representative sample rather than estimating from word counts — particularly for non-English text or code, where the rule of thumb is wrong by a wide margin.

A rough budget before you commit

Anyone building a workflow around a paid model should do this once, on paper, before the first invoice.

Estimate input per call. Instructions plus material plus, for conversations, the accumulated history.

Estimate output per call.

Multiply by expected volume, then by the published rates, remembering that output costs several times input.

Add a factor for retries. Failed calls, regenerated answers, iteration. Two to three times the naive figure is realistic in development.

Then compare with the alternative. A person doing the same task, a simpler method, or a smaller model. Frequently a smaller cheaper model handles the majority of cases with the larger one reserved for the difficult ones, which cuts cost substantially at little quality loss.

Set a hard spending cap with the provider before anything runs unattended. Runaway loops are the standard first-month accident and the cap is the only thing that bounds it.