Context Window
Domain: NLP Fundamentals
🎯 Motivation
The maximum number of tokens a model can see at once. Larger windows mean longer memory but more cost.
📋 Concrete Example
GPT-4 has a 128K token context window. It can read a whole short book, but not a full library.
🔗 Analogy
Like the size of your desk - you can only work with the documents that fit on it at one time.