Skip to main content
Back to catalog

$PostalCodesNL

A Dutch postal-code API fragment that is glitchy across three separate model families — a good demonstration that glitch tokens transfer whenever tokenizers share BPE merges.

Raw token

"$PostalCodesNL"

Observed behaviors

  • 01ChatGPT: unspeakable/polysemantic — variable interpretations and nondeterminism at temperature 0 (both bold-flagged in Watkins' study).LessWrong
  • 02Verified under-trained in Llama-3-8B/70B and Llama-3.1-8B/70B (top examples in the Magikarp reports).GitHub
Observed or documented on
GPT-3.5 / GPT-4Llama 3Qwen
Behavior types
PolysemanticUnder-trained
Tokenizer
cl100k_base / Llama-3 BPE / Qwen BPE
Discovered by
Matthew Watkins (cl100k); Sander Land & Max Bartolo (Llama-3/Qwen)

References

Related tokens

PostalCodesNL$PostalCodesNL