Latency
How long a model takes to respond: the wait for the first token and the speed of the tokens after it.
Also: time to first token, tokens per second, throughput
Users notice two delays. Time to first token is the pause before anything appears, driven by queueing and reading the prompt. Output speed, in tokens per second, decides how fast the rest arrives; comfortable reading needs perhaps 10 to 20, while specialised chips from companies such as Groq and Cerebras have served open models at hundreds or more. Throughput is the total tokens a server produces for all users. Reasoning models add a third delay, the hidden thinking before the visible answer begins.