Mixtral 8x22B --if there is a way to productively downcycle MoEs to dense models, I suspect someone is about to figure out how that works. https://t.co/ccKUthSnAj

Posters: georgejrjrjr. georgejrjrjr

2 likes recorded by the source corpus.