हिंदीखरीदारी

The Devanagari word for “Hindi shopping”, the single most under-trained token in Gemma-7B's vocabulary.

"हिंदीखरीदारी"

Observed behaviors

• 01Rank #1 most under-trained in the Gemma-7B Magikarp repetition verification (E_out cosine-distance indicator 6.56e-06): a max self-repeat probability of just 4.2e-04 — the model effectively cannot say it.GitHub

References

• Magikarp Gemma-7B report — GitHub
• Fishing for Magikarp (arXiv:2405.05417)