Skip to main content
Back to catalog

rawdownloadcloneembedreportprint

The longest glitch token known in r50k — a nested family from website “raw download / clone / embed report print” UI strings (origin analysis by nostalgebraist).

Raw token

"rawdownloadcloneembedreportprint"

Observed behaviors

  • 01GPT-3/ChatGPT truncations produced “embedded” variants: 'embedEMOTE', 'clone this', 'clone my clone', 'embed newcomment'.LessWrong
  • 02Verified under-trained in GPT-2 (all sizes) and GPT-J; family members `rawdownload` and `embedreportprint` are GPT2-xl's farthest-from-centroid tokens.LessWrong
Observed or documented on
GPT-2GPT-3GPT-J-6B
Behavior types
Under-trainedUnspeakable
Tokenizer
r50k_base
Discovered by
Jessica Rumbelow & Matthew Watkins

References