Vocabulary
Domain: Data & Datasets
🎯 Motivation
The set of all tokens a model knows. Larger vocabulary = more tokens = finer text representation.
📋 Concrete Example
Vocabulary: 50,000 tokens. cat, dog, un, believ, able are all in the vocabulary. The model can represent any text using these 50K tokens.
🔗 Analogy
A student’s vocabulary - the words they know. A larger vocabulary means they can express more ideas. A model’s vocabulary works the same way.