have people figured out context caching for LLMs yet? Is it part of any models? as in some way that crunching S+NextToken is way faster if you've already just crunched S
Posters: natural_hazard. natural_hazard
2 likes recorded by the source corpus.
View Post: https://x.com/natural_hazard/status/1844451220905484781