Skip to main content
Back to catalog

SmartyHeaderCode

One of only three Category-A “unspeakable” tokens Adam Yedidia found in the top 2,000 of the cl100k vocabulary; likely a Smarty templating-engine fragment that never appeared in GPT-3.5/4 training data.

Raw token

"SmartyHeaderCode"

Observed behaviors

  • 01GPT-4 mostly treats the token as though it does not exist: it ignores it or responds about something else.LessWrong
  • 02GPT-3.5 invents “creative” interpretations instead of repeating it; making either model spell it out is very hard.LessWrong
  • 03The tests ran on the April 2023 ChatGPT web app (both GPT-3.5 Default and GPT-4 tiers); the author then reproduced it via the API with gpt-3.5-turbo at temperature 0: SmartyHeaderCode came back as “AndHashCode”.LessWrongDemo
Observed or documented on
GPT-3.5 / GPT-4
Behavior types
Unspeakable
Tokenizer
cl100k_base
Discovered by
Adam Yedidia

References