Cordonnier et al.

Explored the equivalence between attention and convolution and empirically found that vision models often have many 2D relative position heads