since it's an MoE I would guess they trained something small and upcycled it into an MoE. looking forward to seeing about its architecture.

Posters: xlr8harder. xlr8harder

2 likes recorded by the source corpus.