“Control tokens” are often implemented in LLMs to separate different types of content (e.g. “assistant”, “user”, “system”).But why are such type tags not instead an additional 3-dimensional one-hot…

Posters: davidad. davidad

4 likes recorded by the source corpus.