GPU Utilization
Domain: MLOps
🎯 Motivation
Maximizing GPU usage during training and inference. Idle GPUs waste money; fully utilized GPUs are efficient.
📋 Concrete Example
Utilization: 95% GPU during training (good). 30% GPU during inference (bad - batch more). Monitor and optimize to keep GPUs busy.
🔗 Analogy
A factory running at 95% capacity - machines are busy, efficient. At 30%, machines sit idle, wasting money. GPU utilization keeps GPUs busy.