@johnowhitaker
This video by @jbhuang0604 manages to cram in all the core pieces of modern attention variants, a perfect refresher if you (like me) need a reminder of the differences between MHA, GQA, MLA, DSA etc :) https://t.co/UrjLHvn1v2 https://t.co/kWlKtfTYkN