Pre-Training
Domain: AI Fundamentals
🎯 Motivation
Training a model on a huge general dataset before fine-tuning on a specific task.
📋 Concrete Example
Pre-train a language model on all of Wikipedia, then fine-tune it for answering science questions.
🔗 Analogy
Like a student doing general reading for years before specializing in a university major.