MultiheadAttention conversion #49
Description
Hello, I use MultiheadAttention in my model, how do I bring in agent-attention? It's best to do it without retraining the model. My idea is to replace the computational part of q,k,v in its forward propagation, is that possible?