␣ForCanBeConverted

A C#-compiler-flavored fragment (“can be converted to foreach”) that became the most famous “polysemantic” glitch token: the model perceives it as a different word every time. It exists in cl100k, Llama-3 and Qwen vocabularies because the tokenizers share BPE merges.

" ForCanBeConverted"

␣marks a real space; the symbol itself is not part of the token, and copying preserves it.

Observed behaviors

• 01ChatGPT/gpt-3.5-turbo interprets it as a different word on every attempt, even re-running the identical prompt — a different message every time at temperature 0.LessWrong
• 02Verified under-trained in Llama-3-8B/70B, Llama-3.1-8B/70B and multiple Qwen models in the Magikarp repetition-verification reports.GitHub
• 03ChatGPT sometimes gets stuck in loops or terminates the message at the token (suggesting it is treated as a begin/end-of-sequence marker).LessWrong

References

• A Search for More “Unspeakable” Glitch Tokens — LessWrong
• Magikarp results summary — GitHub

Related tokens