Multi-Head Attention
Domain: LLM Concepts
🎯 Motivation
Words relate to each other in multiple ways simultaneously - grammar, meaning, emotion. Multiple attention heads capture different relationships at once.
📋 Concrete Example
One head might track grammar (subject-verb), another tracks emotion (positive/negative words), another tracks entities (person names). Together they give a richer understanding.
🔗 Analogy
A group of detectives working the same case - one examines fingerprints, another interviews witnesses, another analyzes finances. Each focuses on different evidence; together they solve the case.