Home

Benchmarks

Discover open, rigorous benchmarks and leaderboards from top AI labs and researchers, all in one place.

Featured Benchmark

Explore a rotation of featured benchmarks.

Thiomi ASR/TTS Baseline

Thiomi NLP; Harvard UniversityExtrinsic · 2 tasks · 10 languagesPublished 2026
LanguageMasai (Maa) (mas)
TaskASR/TTS, N/A
MetricsWER/MOS
LicenseCC-BY

SOTA: ASR: 14.2% WER; TTS: 3.41 MOS

View Benchmark

Arab Voices: Standardized Dialectal Arabic ASR Benchmark

Authors not fully captured in registry; Arab Voices paper teamExtrinsic · 1 task · Multiple Arabic dialects; aao not included languages
LanguageAlgerian Saharan Arabic (aao)
TaskASR / dialectal Arabic speech evaluation
MetricsWER, CER
LicenseUnknown

SOTA: Dialectal Arabic ASR benchmarking

View Benchmark

Arab Voices: Standardized Dialectal Arabic ASR Benchmark

arXivExtrinsic · 1 task · Multiple Arabic dialects including Hejazi Arabic / acw languagesPublished 2026
LanguageHejazi Arabic (acw)
TaskASR, Speech Recognition / Encoder evaluation
MetricsWER, CER
LicenseUnknown

SOTA: Dialectal Arabic ASR evaluation

View Benchmark

EthioMT Benchmark (MT for Ethiopian Languages)

Atnafu Lambebo Tonja; Olga Kolesnikova; Alexander Gelbukh; Jugal KalitaExtrinsic · 1 task · 23 languagesPublished 2024
LanguageAfar (aar)
TaskMT, N/A
MetricsBLEU
LicenseCC-BY 4.0

SOTA: First MT benchmark results for Afar (aar)

View Benchmark

AfroLingu-MT Benchmark

Elmadany, Abdelrahim; Adebara, Ife; Abdul-Mageed, Muhammad (UBC-NLP)Extrinsic · 1 task · 46 languagesPublished 2024
LanguageAfar (aar)
TaskMT, Encoder-Decoder
MetricsspBLEU_1K
LicenseUnknown

SOTA: Toucan MT model SOTA on African language pairs including aar

View Benchmark

HornMT Evaluation Benchmark

Asmelash Teka Hadgu; Gebrekirstos G. Gebremeskel; Abel Aregawi (Lesan AI)Extrinsic · 1 task · 5 languagesPublished 2022
LanguageAfar (aar)
TaskMT, N/A
MetricsBLEU
LicenseCC-BY 4.0

SOTA: Standard MT benchmark for Horn of Africa languages including aar

View Benchmark

Community Benchmarks

AfroBench

Ojo, Jessica; Ogundepo, Odunayo; Oladipo, Akintunde; et al. (McGill NLP / Masakhane)Multi-task · 15 tasks · 64 languagesPublished 2025
LanguageAfar (aar)
TaskMulti-task NLU/NLG, N/A
MetricsAccuracy/F1/BLEU/ROUGE
LicenseUnknown

SOTA: Multi-task LLM evaluation for 64 African languages

View Benchmark

GlotLID-M language identification evaluation for Adioukrou

CIS LMU Munich / GlotLID authorsIntrinsic · 1 task · Multiple languages including Adioukrou adj languagesPublished 2023
LanguageAdjoukrou (adj)
TaskLanguage Identification, Classification / LID evaluation
MetricsF1, FPR
LicenseUnknown

SOTA: Language Identification

View Benchmark

ASJP Putukwam lexical comparison resource

ASJP Consortium; compiled by Tessa de Mol-van ValenIntrinsic · 1 task · 1 language
LanguagePutukwam (afe)
TaskOther / lexical comparison
MetricsDistance / lexical similarity possible; no standard NLP metric reported
LicenseCC-BY-4.0
View Benchmark

FLORES-200

Meta AI (NLLB Team)Extrinsic · 1 task · 200 languagesPublished 2022
LanguageAcholi (ach)
TaskMT
MetricsBLEU, chrF, XSTS
LicenseCC-BY-SA

SOTA: MT across 200 languages; Acholi included

View Benchmark

SALT MT Benchmark

Sunbird AI + MakerereExtrinsic · 1 task · 6 (Ugandan langs + English) languagesPublished 2022
LanguageAcholi (ach)
TaskMT
MetricsBLEU, chrF
LicenseCC-BY

SOTA: MT (Ugandan languages)

View Benchmark

Sunflower MT Benchmark

Sunbird AIMulti-task · 3 tasks · 31 + English languagesPublished 2025
LanguageAcholi (ach)
TaskMT, ASR, Knowledge QA / Causal LM
MetricsBLEU, chrF, WER, CER, Accuracy
LicenseCC-BY

SOTA: Sunflower-32B: highest mean chrF (0.435) local-to-English across 31 langs

View Benchmark