GPT-3

The canonical glitch token: the Reddit handle of a prolific r/counting user. It was scraped into the tokenizer-training corpus but rarely seen in actual model training data, leaving its embedding essentially untrained.

Almost certainly the handle of Bitcoin developer Peter Todd: tokenized from web data but under-trained, and strongly entangled with crypto/AI associations.

Reddit handle of another r/counting “Hall of Counters” member (a fan of game developer Nitrome). The only token family whose owner publicly acknowledged their tokenization.

Reddit handle of a second r/counting counter (“a pretty random handle”), scraped from the same Hall-of-Counters chart as SolidGoldMagikarp.

Truncated handle of Redditor davidjl123, another r/counting counter. Unusually, it is anomalous under both r50k and the newer cl100k tokenizer.

Reddit handle of another of the six prolific r/counting counters.

Name of a bot built by the Twitch Plays Pokémon community that auto-posted chat messages to a Reddit live-updater thread; confirmed by its creator “Sparkette” in the LessWrong comments.

Originates from Rocket League crash logs full of lines like `Message=PsyNetMessage_X_57`, which were heavily posted to Reddit and scraped.

A Kerbal Space Program identifier (attachment rotation) from KSP save/craft files posted online; one of about ten glitch tokens traced to KSP.

GUI-state identifier from the same KSP data dump as attRot, alongside ` strutConnector`, ` guiIcon`, ` srfAttach` and others.

Another Kerbal Space Program token; also the single closest-to-centroid token in GPT2-small (distance 1.5305).

Fragment of the e-commerce backend field “BuyableInstoreAndOnline” (traced to an abandoned Weebly shop's HTML); the same source yielded `quickShip`, `isSpecialOrderable`, `wcsstore` and others.

The longest glitch token known in r50k — a nested family from website “raw download / clone / embed report print” UI strings (origin analysis by nostalgebraist).

Classic mojibake token: UTF-8-misdecoded Ã/Â sequences repeated, merged into a single BPE token because such mis-encoded text was frequent in the tokenizer corpus but absent from model training. A 16-repetition variant and `ÛÛ` belong to the same family.

A token of completely unknown origin — un-Googleable even in quotes — that became famous for triggering the most aggressive documented completion.

A truncated variant of ␣SolidGoldMagikarp (missing the “Solid”): the same r/counting handle fragment, scraped into the tokenizer corpus but barely seen in model training data.

Handle fragment of a Reddit r/counting user, swept into the vocabulary in the same batch as SolidGoldMagikarp but left under-trained.

Fragment of the name of “StreamerBot”, a Twitch streaming bot.

A substring token of ` petertodd`: under-trained in its own right and inheriting the parent token's strange aura.

Handle fragment of Bitcoin Core developer Greg Maxwell — a “Bitcoin celebrity” glitch token alongside ` petertodd`.

A fragment of the space-sandbox game Space Engineers' name — the token most often uttered “on behalf of” others in the inter-referentiality network.

Fragment of a Puzzle & Dragons character name (the 「龍喚士」 / Dragonbound series).

Fragment of the Puzzle & Dragons character Leilan's name — the most popular stand-in in the petertodd transposition experiments.

Fragment of the Puzzle & Dragons character Skydragon's name.

The Japanese katakana 「ゼウス」 (Zeus of Greek myth). An ordinary word turned glitch token, presumably under-trained because this exact form was rare in training data.

The katakana 「サーティ」 (“thirty”), traced to a Puzzle & Dragons × Baskin-Robbins (“31 Ice Cream” in Japan) collaboration character name.

Fragment of the e-commerce inventory field “BuyableInstoreAndOnline”, from the same source as `oreAndOnline`.

Fragment of the academic-slideshow script “View large Download” (origin traced in SGM3).

A log fragment of Minecraft's Forge mod loader.

A Minecraft multiplayer-server log fragment (the MpServer class name).