Darkwood · Symfony · Flow · language

Nolife Language

Don't say cost. Say investment.

When words change both persuasion and token cost.

Words change the frame

COST

↓

INVESTMENT

basic → essential

standard → customized

few → limited

Same offer. Different frame.

Then the tokenizer reads it

Humans hear meaning.
Tokenizers count pieces.

words ≠ tokens

chars ≠ tokens

encoding matters

Three axes

meaning

↓

token cost

+

persuasion

Improving one axis can hurt another.

o200k_base · parallel corpus

Same idea. Three languages.

English

46

baseline

French

58

+26.09%

German

63

+36.96%

this corpus × this tokenizer — not a universal language tax

Same texts · two encodings

Language cost is not a constant.

	o200k_base	cl100k_base
English	46	46
French	58	71
German	63	73

FR +26.09% → +54.35%

DE +36.96% → +58.70%

corpus × tokenizer × language

o200k_base · German persuasion

Standard

1 token

→

maßgeschneidert

5 tokens

+400%

Basis 1 → unverzichtbar 4

Better sales framing can be worse BPE.

Four corners of the experiment

A phrase can win one axis and lose another.

stronger frame · fewer tokens

$1,000 → $999

4 → 2

different frame · same tokens

cost → investment

1 → 1

stronger frame · more tokens

Standard → maßgeschneidert

1 → 5

simpler frame · fewer tokens

maßgeschneidert → passend

5 → 2

Regressions

9 / 27 rewrites got more expensive

all 9 were persuasion · 0 were token_opt

Original	Replacement	Before	After
Basis	unverzichtbar	1	4
Standard	maßgeschneidert	1	5
Kosten	Investition	1	2
achetez maintenant	plus que trois en stock	3	5

A benchmark that only shows green rows is a brochure.

Mixed pair aggregate

62 → 62

Zero change in the total. Huge change underneath.

persuasion

36 → 45

+9 tokens

token_opt

26 → 17

−9 tokens

Cancellation ≠ neutrality.

ESTIMATED · 1,000,000 requests · pair tokens

The money is tiny. The structure is not.

persuasion

input −$1.35

output −$5.40

token_opt

input +$1.35

output +$5.40

mixed

$0.00

tiny corpus · dated gpt-4o-mini snapshot · not a provider invoice

100,000-token hypothetical window · o200k_base

Same window. Less language.

English

2,173.91

copies

French

1,724.14

−20.69%

German

1,587.30

−26.98%

The model does not get a smaller context window.
The same token budget fits less of this text.

Caveman · method, not a rerun

Compression has overhead too.

shorter output  ≠  lower total cost

publish regressions

compare against a control

output ≠ invoice

Inspected as methodology. Not a second benchmark on this corpus.

content/nolife-language

Seven jobs. One measurement.

1LoadCorpus

2TokenizeBefore

3ApplyCandidates

4TokenizeAfter

5Compare

6EstimateCost

7Report

Symfony 8.1

PHP 8.5.4

darkwood/flow 8.1.6

yethee/tiktoken 1.1.1

php bin/console app:language:benchmark

What I learned

Natural languages are not tokenization-equivalent

Tokenizer choice changes the gap

Persuasion and token efficiency can conflict

Character count is not token count

Aggregates can hide opposing movements

Measure the actual workload before you optimize it.