Languages
/
Zulu

13.4% fewer tokensfor Zulu text.

Nzinga needs 39,850 tokens for 1,012 FLORES+ sentences. GPT-4o, the best other, needs 46,007.

Zulu needs 13.4% fewer tokens than GPT-4o, the best other tokenizer.
13.4%
fewer tokens
1.47x
premium on FLORES+
25.0M
characters
12.5%
saving on NTREX-128

The challenge

Zulu text needs more tokens than its English source. With Gemma 3, the same 1,012 FLORES+ sentences take 53,281 tokens, against 27,114 in English.

The change

Nzinga added 32,000 word pieces to Gemma 3's tokenizer, built from open web text with 25.0 million characters of Zulu. English barely changes.

flores+: nzinga: 39,850 gpt-4o: 46,007 gemma-3: 53,281

Zulu is inside the 1.3 to 1.5 target.

FLORES+ devtest
,
premium 1.47x
,
Zulu

The result

Nzinga needs 13.4% fewer tokens than GPT-4o on FLORES+ and 12.5% fewer on NTREX-128. The premium falls to 1.47 times English, inside the target.

Before Nzinga, and after.

Before: Gemma 3 alone
Tokens for 1,012 FLORES+ sentences
Before
53,281
After
39,850
Tokens on NTREX-128 news
Before
92,136
After
70,509
Premium on NTREX-128 news
Before
1.74x
After
1.33x

More languages

All languages
Pixel mosaic cover image for the Yoruba result
NTREX-128
·
Yoruba
Yoruba needs 24.7% fewer tokens than Aya Expanse on NTREX-128, though above the target.
Pixel mosaic cover image for the Kinyarwanda result
FLORES+
·
Kinyarwanda
Kinyarwanda premium falls to 1.56 times English, from 1.71 with GPT-4o.
Pixel mosaic cover image for the Hausa result
FLORES+
·
Hausa
Hausa needs 6.6% fewer tokens than GPT-4o on FLORES+ devtest.
Closing the gap

Fewer tokens for African-language AI models.

Record in your language. The demo is open.

Request an invite
Try the app