Pre-Training
Domain: NLP Fundamentals
🎯 Motivation
Training a language model on a massive text corpus before any specific task. Gives general language understanding.
📋 Concrete Example
GPT is pre-trained on web text, books, and code. Then it is fine-tuned or prompted for specific tasks.
🔗 Analogy
Like a student doing broad reading for years before specializing in one subject at university.