>20k QPS MQA everywhere local attention, 1/6 layers use global attn KV cache tied between layers, cached on host memory with "95% cache rate" trained in int8 precision https://t.co/7mrMyHburk
Posters: cis_female. cis_female
3 likes recorded by the source corpus.
View Post: https://x.com/cis_female/status/1803816791677808964