\u200Cآمباردا

A Persian fragment beginning with a zero-width non-joiner (U+200C ZWNJ, an invisible character); the #2 most under-trained token in Gemma-7B.

"‌آمباردا"

Observed behaviors

• 01Rank #2 most under-trained in the Gemma-7B Magikarp repetition verification (E_out cosine-distance indicator 7.39e-06): a max self-repeat probability of just 4.4e-04.GitHub

References

• Magikarp Gemma-7B report — GitHub
• Fishing for Magikarp (arXiv:2405.05417)