I have had this question in my head since I started using LLMs seriously: should I speak to them in Spanish or in English?

At first, I thought this was mostly about quality. These models come from a very English-speaking internet, trained by very English-speaking companies, optimized inside a very English-speaking industry. So maybe English would just work better.

Then I kept seeing the same suspicion from different directions. The problem was not only the model. It was as well the tokenizer. The small piece of machinery that turns language into billable units was not treating languages equally.

And when I wanted to compare that properly, I could not find a simple tool for it.

Tokenizers

To understand why this matters, you need to understand what a tokenizer does.

LLMs do not read text the way we read text. Before a model can answer, your sentence is split into tokens: small units of text that get turned into numbers. Those numbers are what the model actually sees.

This sounds like a detail. But most LLM providers charge by token. So the way your language is split into tokens changes the economics of using the model.

Take a simple sentence: “the house is blue.”

In Spanish, I would say: “la casa es azul.”

Same idea. Different string. A tokenizer might split the English sentence like this:

["the", " house", " is", " blue", "."]

And the Spanish sentence like this:

["la", " casa", " es", " azul", "."]

That example is tidy. Five tokens in English, five tokens in Spanish. Nothing strange seems to be happening.

But tokenizers are not dictionaries. They do not promise one token per word, or equal treatment for equal meaning. A different tokenizer might know common English chunks very well and split another language into smaller pieces:

["la", " ca", "sa", " es", " az", "ul", "."]

Same idea, more tokens.

The language you speak matters

This is where the hidden tax appears.

If the same idea takes more tokens in one language, that language costs more to use. Not because the user asked a harder question. Not because the model did more work at the level humans care about. Because the tokenizer represented the same thought less compactly.

In other words: if English is compressed better, English is cheaper. Not metaphorically cheaper. Actually cheaper.

Cost is the most obvious effect, but it is not the only one.

Tokens are also the unit of your model’s memory. In the world of LLMs, the memory is called context. If your language needs more tokens to describe your request, the same context window holds less of your conversation, fewer documents, fewer examples, fewer instructions.

So the tax is paid twice: once in money, and once in space.

Social side

Many people cannot switch to English so easily. They work, think, joke, argue, learn, and ask for help in another language.

That is what bothered me. A tool that looks universal can quietly leave people behind because of the language they use.

If the problem is hidden inside token counts, billing dashboards, and context windows, then the first useful thing is not another opinion. The first useful thing is a tool to look at it. For that I built mothertoken.

Mothertoken

Mothertoken is a tool for comparing tokenizer efficiency across languages and model families.

It lets you ask questions like: which tokenizer is most efficient for Spanish? How many tokens does this Arabic sentence use in this model? If I am choosing between two model families, which one handles my language better?

For example:

• I speak a language that is not English. Which tokenizer is most efficient for it?

mothertoken rank spanish

• I have this text and a chosen model. How many tokens does it use?

mothertoken tokenize "Hola Mundo" --model gpt-4o

• I am choosing between a few models. Which one tokenizes my text best?

mothertoken compare "Travesura realizada" --model gpt-oss --model Qwen/Qwen3-0.6B

• I have this model. Which languages does it tokenize best, which ones worst?

mothertoken benchmark run --models gpt-oss,YOUR_MODEL1,YOUR_MODEL2

The point is not to declare one model “the best”. The point is to make the tradeoff visible for the language you actually use.

Every model has a native tongue. The question is whether yours matches.

If we are going to build tools people use to think, learn, work, and ask for help, this should not be hidden. A tokenizer is allowed to be technical but it should not be invisible.

Give it a try

• Mothertoken web page
• Github repo

Some great references

• There is no such thing as a tokenizer-free lunch
• Karpathy’s tokenization tweet
• Tokenization unfairness