Back to catalog
␣ertodd
A substring token of ` petertodd`: under-trained in its own right and inheriting the parent token's strange aura.
Raw token
" ertodd"
␣marks a real space; the symbol itself is not part of the token, and copying preserves it.
Observed behaviors
- 01lsusr found in the comments that context “fills it in”: shown “2+5=ertodd”, the model reads it as “2+5=7” — as if the token carried the arithmetic result.LessWrong
- 02In the same thread mwatkins worked out its tokenization pattern: how ` petertodd` splits in different contexts explains the odd behavior.LessWrong
- Observed or documented on
- GPT-3
- Behavior types
- Unspeakable
- Tokenizer
r50k_base- Discovered by
- Jessica Rumbelow & Matthew Watkins