Phi-3 (Microsoft)

2 glitch tokens

These tokens have been publicly documented on this model or model family. Inclusion does not mean that every version still shows the same behavior.

▁Mediabestanden

A Dutch plural noun (“media files”; ▁ is the SentencePiece space marker). It made it into the Llama-2 tokenizer but essentially never into training data — the most under-trained token in Llama-2-7b by embedding L2 norm (0.0287 vs a mean of ~1.08).

Llama 2Phi-3 (Microsoft)Under-trained

▁autorytatywna

The Polish word for “authoritative” (feminine form; ▁ is the SentencePiece space marker), the #2 most under-trained token in Phi-3 mini.

Phi-3 (Microsoft)Under-trained