Have you ever wondered why transformers use separate learned projections for Q, K, and V in their attention mechanism? It seems like a redundant design choice, especially since Q is only used to match against K, and V is just the ‘payload’ we sum using attention weights. Why not simplify the design by setting Q = X and V = X, and only learning W^K to produce the keys?
Well, the reason is that separate projections allow the model to learn more nuanced and context-dependent representations. By learning separate projections, the model can capture different aspects of the input data, which is especially important in natural language processing tasks where context is everything.
Tying Q and V directly to the input embeddings would limit the model’s ability to learn these nuanced representations, which could negatively impact its performance. Additionally, separate projections allow the model to generalize better to new, unseen data.
So, while it may seem like an unnecessary complexity, the use of separate projections for Q, K, and V is a key component of the transformer’s success.