Next Token Prediction
Domain: NLP Fundamentals
🎯 Motivation
The fundamental task of autoregressive language models: guess the next token from the context so far.
📋 Concrete Example
Input: Hong Kong is famous for its. Model predicts: dim sum (0.3), skyline (0.2), harbor (0.15), …
🔗 Analogy
Like a guessing game where you predict the next word in a sentence your friend is saying.