Run
~/african-tokenizer
$
python scripts/flores_tier1.py
✓ languages 12 Tier 1
✓ 13-gram overlap 0 of 1,012
● written results/flores-tier1.json
Nzinga · results
tier1.premium
Tier 1 premium
1.55x
best.saving
Fewer than the best other
22%
languages.ahead
Ahead, of 21
21
Overview
FLORES+ is the Open Language Data Initiative's test set, derived from FLORES-200. Nzinga uses devtest: 1,012 professionally translated sentences per language, checked for 13-word overlap with training text.
What it measures
devtest
split
1,012 translated sentences in each language
tier1
set
12 Tier 1 languages, English and French
premium
ratio
Tokens against the English sentences
overlap
check
No 13-word sequence in training
coverage
count
21 of the 26 NTREX languages
Setup
01
Accept the dataset terms
FLORES+ is gated on Hugging Face; accept its terms first.
02
Set your token
Export HF_TOKEN so the script can download the devtest split.
03
Run the benchmark
Run flores_tier1.py. Every token count is written to the results folder.
Count your own sentence in the browser.
Paste a sentence into the live comparison. Every figure here is read from a results file.
$ bash scripts/fetch_data.sh
$ export HF_TOKEN=hf_••••••••
$ python scripts/premium_v2.py
✓ NTREX-128 · ✓ FLORES+ · ✓ baselines
● 5 results files · about 4 min