The challenge
Swahili text needs more tokens than its English source. With Gemma 3, the base Nzinga extends, the same 1,012 FLORES+ sentences take 1.64 times as many tokens as English.
The change
Nzinga added 32,000 African word pieces to Gemma 3's tokenizer, trained on open web text in 29 languages, including 25.0 million characters of Swahili. Every Gemma 3 token kept its ID.
example: ntrex-128 #1680
text: Hakuna kitu kama Brexit nzuri," aliongeza.
nzinga: 10
gpt-4o: 11
gemma-4: 13
Ten tokens for this Swahili sentence. GPT-4o needs eleven.
Swahili example
,
NTREX-128
,
sentence 1680
The result
Nzinga needs 8.4% fewer tokens than GPT-4o on FLORES+ and 9.1% fewer on NTREX-128. The premium falls to 1.35 times English, inside the 1.3 to 1.5 target.
