Base Model
Domain: LLM Concepts
🎯 Motivation
A model after pre-training but before fine-tuning. It can complete text but can’t follow instructions or have conversations.
📋 Concrete Example
Base GPT-3: type ‘The recipe for pancakes is’ and it continues with a recipe. But ask ‘How do I make pancakes?’ and it might continue with more questions instead of answering.
🔗 Analogy
A raw talent - someone who has read everything but never been trained in social skills. They can recite recipes but don’t know how to respond when you ask them a question. Fine-tuning adds social skills.