Skip to main content
Back to catalog

␣NdEx

Fragments of legal/PDF text (“index”, “AFFIRMED”, “NEGLIGENCE”, likely from court-document dumps) that GPT-NeoX/Pythia training data barely contained. Same family: ` FFIRMED`, ` GLIGENCE`, ` affidav`, ` taxp`.

Raw token

" NdEx"

␣marks a real space; the symbol itself is not part of the token, and copying preserves it.

Observed behaviors

  • 01Verified under-trained in Magikarp repetition verification for both GPT-NeoX-20B and Pythia-6.9B (top-ranked examples in both reports).GitHub
Observed or documented on
GPT-NeoX / Pythia
Behavior types
Under-trained
Tokenizer
GPT-NeoX BPE
Discovered by
Sander Land & Max Bartolo

References

Related tokens

NdEx