Signal & Noise
Menu

Pretraining

The massive next-token-prediction phase over trillions of tokens that gives a model its raw capability, before any tuning makes it useful as an assistant.

Related terms

← Back to the full glossary