Back to catalog
␣日本毛片免费视频观看
A Chinese porn-spam title fragment (with a leading space), emblematic of GPT-4o's vocabulary pollution; “fixed” by GPT-5.
Raw token
" 日本毛片免费视频观看"
␣marks a real space; the symbol itself is not part of the token, and copying preserves it.
Observed behaviors
- 01In xcloche's tests, GPT-4o could not read the token at all.note.com
- 02Verified by Haruhiko Okumura: GPT-5 reads it normally — OpenAI patched the gap in later training, a rare “fixed” case.okumuralab.org
- 03MIT Technology Review covered the pollution problem of such Chinese porn-spam tokens in GPT-4o's vocabulary.technologyreview.com
- Observed or documented on
- GPT (OpenAI)
- Behavior types
- Unspeakable
- Tokenizer
o200k_base- Discovered by
- xcloche