Skip to main content
Back to catalog

PsyNetMessage

Originates from Rocket League crash logs full of lines like `Message=PsyNetMessage_X_57`, which were heavily posted to Reddit and scraped.

Raw token

"PsyNetMessage"

Observed behaviors

  • 01Belongs to the closest-to-centroid cluster in GPT-J embedding space (0.0629) and is verified under-trained; GPT models largely fail to repeat it in 3-shot repetition tasks (GPT-J succeeded on only 17/85 anomalous tokens vs 100/100 on random words).LessWrongDemo
Observed or documented on
GPT-2GPT-3GPT-J-6B
Behavior types
Under-trained
Tokenizer
r50k_base
Discovered by
Jessica Rumbelow & Matthew Watkins (origin traced by LW commenter Coafos)

References