I think this is true, important, and mostly misunderstood, including by many experts. FFN tensors are frozen; KV tensors grow during inference (though append-only). It’s unconventional to call KV…

Posters: davidad. davidad

2 likes recorded by the source corpus.

Topics: Software, AI & LLMs

View Post: https://x.com/davidad/status/1958148520872948094