Throughput
Domain: Evaluation
🎯 Motivation
How many requests the model can handle per second. Critical for serving many users.
📋 Concrete Example
Throughput: 1000 requests/second. If 5000 users ask simultaneously, some wait. Higher throughput = more simultaneous users served.
🔗 Analogy
A highway’s capacity - how many cars per hour. More lanes = higher throughput. Model throughput is how many requests it handles simultaneously.