Token
Everyone says AI pricing is based on tokens. What is a token, and how does it relate to what I type?
Everyone says AI pricing is based on tokens. What is a token, and how does it relate to what I type?
usage = input + outputThis illustrates how chunks relate; it is not an exact Token result for any specific model.
Split into rule-based pieces: The model breaks sentences into tokens using its tokenizer. English is roughly 1 token per word; Chinese is around 1.5–2 tokens per character.
Generated piecewise, counted both ways: It reads input pieces and generates output pieces one at a time. Both your prompt and the generated response count toward token usage.
Keep files relevant to protect quality and budget: A webpage's source is roughly 2,000–5,000 tokens. Sending only relevant files instead of entire projects saves cost and avoids diluting model focus.
Report input, output, and cached tokens for the current model, including system rules, history, files, and tool results in input. Do not convert directly from character count; explain context usage and the cost basis.