TTFT / TPS
You might say
Why do some AI tools take forever to show the first word while others answer right away?
Measures of time to first output and of token generation speed after output beginsTTFT measures time from sending a request to receiving the first token. TPS measures tokens produced each second afterward. A quick first token does not guarantee a short total response time.
Know first
TTFT & TPS
Where do three kinds of wait come from on one timeline?
These times and speeds are illustrative; they only compare how the metrics relate.
Send requestReceive first TokenAnswer complete
Total time 6.2sB starts speaking sooner, but generates more slowly afterward, so its full answer finishes later.
Separate first output, speed, and total time
Wait for TTFT：TTFT runs from send to the first text token.
Then watch TPS：TPS is tokens per second after generation begins, not transactions per second.
Then measure total time：A longer answer can still take longer to finish even when it begins quickly.
Learn next
Further reading