Pre-Training
Domain: LLM Concepts
🎯 Motivation
Before an LLM can answer questions, it needs to learn language patterns from massive text data. Pre-training is that initial learning phase.
📋 Concrete Example
The model reads billions of web pages, books, and articles, learning to predict the next word. After pre-training, it knows language patterns but can’t follow instructions yet.
🔗 Analogy
A baby listening to people talk for years before speaking - they absorb patterns, sounds, and grammar. They’re learning the building blocks (pre-training) before they can have a real conversation.