GPT-2

The canonical glitch token: the Reddit handle of a prolific r/counting user. It was scraped into the tokenizer-training corpus but rarely seen in actual model training data, leaving its embedding essentially untrained.

Almost certainly the handle of Bitcoin developer Peter Todd: tokenized from web data but under-trained, and strongly entangled with crypto/AI associations.

Reddit handle of another r/counting “Hall of Counters” member (a fan of game developer Nitrome). The only token family whose owner publicly acknowledged their tokenization.

Reddit handle of a second r/counting counter (“a pretty random handle”), scraped from the same Hall-of-Counters chart as SolidGoldMagikarp.

Truncated handle of Redditor davidjl123, another r/counting counter. Unusually, it is anomalous under both r50k and the newer cl100k tokenizer.

Reddit handle of another of the six prolific r/counting counters.

Name of a bot built by the Twitch Plays Pokémon community that auto-posted chat messages to a Reddit live-updater thread; confirmed by its creator “Sparkette” in the LessWrong comments.

Originates from Rocket League crash logs full of lines like `Message=PsyNetMessage_X_57`, which were heavily posted to Reddit and scraped.

A Kerbal Space Program identifier (attachment rotation) from KSP save/craft files posted online; one of about ten glitch tokens traced to KSP.

GUI-state identifier from the same KSP data dump as attRot, alongside ` strutConnector`, ` guiIcon`, ` srfAttach` and others.

Another Kerbal Space Program token; also the single closest-to-centroid token in GPT2-small (distance 1.5305).

Fragment of the e-commerce backend field “BuyableInstoreAndOnline” (traced to an abandoned Weebly shop's HTML); the same source yielded `quickShip`, `isSpecialOrderable`, `wcsstore` and others.

The longest glitch token known in r50k — a nested family from website “raw download / clone / embed report print” UI strings (origin analysis by nostalgebraist).

Classic mojibake token: UTF-8-misdecoded Ã/Â sequences repeated, merged into a single BPE token because such mis-encoded text was frequent in the tokenizer corpus but absent from model training. A 16-repetition variant and `ÛÛ` belong to the same family.

A token of completely unknown origin — un-Googleable even in quotes — that became famous for triggering the most aggressive documented completion.

A truncated variant of ␣SolidGoldMagikarp (missing the “Solid”): the same r/counting handle fragment, scraped into the tokenizer corpus but barely seen in model training data.

A fragment of the space-sandbox game Space Engineers' name — the token most often uttered “on behalf of” others in the inter-referentiality network.

Fragment of the e-commerce inventory field “BuyableInstoreAndOnline”, from the same source as `oreAndOnline`.