Character-Level Tokenization
Domain: NLP Fundamentals
🎯 Motivation
Splits text into individual characters. No out-of-vocabulary problem, but sequences get very long.
📋 Concrete Example
AI becomes [A, I]. Every character is a token.
🔗 Analogy
Like spelling out a word letter by letter - no word is unknown, but it takes a long time to say.