Research

the tokenizer keeps its own count

Hinglish textese saved a few tokens and lost some meaning

7 min read