Chinchilla scaling laws look VERY wrong to me when I try to apply them to the loss curves in the Llama paper, and compute optimal training would be >> 20 tokens/parameter. I have a gigantic…

Posters: georgejrjrjr. georgejrjrjr

2 likes recorded by the source corpus.

Topics: AI & LLMs

View Post: https://x.com/georgejrjrjr/status/1649285033289555968