Open — attribution required; commercial use allowed.
Every column the registry tracks. Values marked derivedwere filled by a rule or read from the source's text rather than stated in its metadata; the rule or the quoted sentence is shown under the value.
Record indexed from Hugging Face
22 May 2025
Source status: Active
22 May 2025
This is the same multilingual corpus, sliced for other languages the platform indexes it under.
Data format: parquet
Used in 200-language MT benchmark
Data format: parquet
Moroccan Arabic in 200-language MT benchmark
Data format: parquet
Data format: parquet
"Multilingual MT evaluation corpus includes Bemba"
"Language code bez represented in FLORES-200"
Bari partially covered in FLORES under Nuer/Bari cluster; verify nus_Latn
Chokwe (cjk_Latn) in 200-language MT benchmark
Guinea-Bissau Creole (cpp_Latn) in 200-language MT benchmark
FLORES-200 parallel MT evaluation corpus includes Southwestern Dinka (dik_Latn). 3001 sentences, dev/devtest splits. First large-scale MT eval coverage for d…
No Language Left Behind seed data with dyu coverage for machine translation
Data format: parquet
Data format: parquet
Exact licence is CC-BY-SA-4.0, which is not available in the requested licence dropdown; dev and devtest total 2,009 sentences.
Data format: parquet
Data format: parquet
Data format: parquet
Data format: parquet
Aggregation of MasakhaNER 1.0 + 2.0 in byte-level span format; Ghomálá' (bbj) included; EMNLP 2022 aclanthology:2022.emnlp-main.298
Largest MT benchmark for African languages in news domain; 21 languages incl. Ghomálá' (bbj); French/English source; used as monolingual source for MasakhaNE…
POS tagging dataset in 20 African languages incl. Ghomálá' (bbj); train 750 / dev 149 / test 599; 17 UD POS tags; arXiv:2305.13989