Prefill
The parallel processing of the entire prompt before generation begins. Compute-bound (vs memory-bound decoding); its duration sets time-to-first-token.
The parallel processing of the entire prompt before generation begins. Compute-bound (vs memory-bound decoding); its duration sets time-to-first-token.