Token
You might say
Everyone says AI pricing is based on tokens. What is a token, and how does it relate to what I type?
A small piece of text or code that a model reads and writesModels split text, punctuation, and code into tokens using their own rules. Input and output token counts affect available context, wait time, and cost; a token is not always one word or one character.
Know first
How a mixed-language sentence becomes tokens
Your inputFix loginButton.tsx
1Fix2fix3 login4Button5.6tsx
Input 6 illustrative piecesOutput 3 illustrative pieces
usage = input + outputThis illustrates how chunks relate; it is not an exact Token result for any specific model.
How text becomes pieces a model processes
Split the text：The model breaks a sentence into text, punctuation, or code pieces using its own tokenizer.
Read and write：It reads input pieces and produces output pieces one at a time.
Count both sides：Both what you send and what the model returns count toward usage.
Learn next
Further reading