Datasets
DATASETHugging Face
5/5

UDHR-LID Language Identification Dataset (CIS-LMU)

Dagbani(dag)TextLanguage IDWest Africa
View Source

Dataset Info

Language
Dagbani (dag)
Modality
Text
Domain
General
Size
10K<n<100K
Institution
CIS-LMU (LMU Munich)
Last Updated
27 Oct 2026

License & Access

SPDX identifierCC0-1.0

Public domain — no rights reserved; unrestricted use.

RedistributionYes

PII RISK

Unknown risk

Full Record

28 of 46 fields stated

Every column the registry tracks. Values marked derivedwere filled by a rule or read from the source's text rather than stated in its metadata; the rule or the quoted sentence is shown under the value.

Registry columns (20/27)

iso code
dag
Dataset name
UDHR-LID Language Identification Dataset (CIS-LMU)
Modality
Text
Type
dataset
Link
https://huggingface.co/datasets/cis-lmu/udhr-lid
NLP Task
Language IDderivedkeyword 'LID' in the title
Domain
General
Data Format
csv
Size Metric
n (samples)
Size Value
10K<n<100KderivedHugging Face card: size_categories
License
CC0-1.0
Open Access
Open
Source Repository
Hugging Face
Year Published
2023
Creator/Author(s)
CIS-LMU (LMU Munich)
Funder/Partner
Not stated
Dialect Coverage
Standard
Gender Balance
Not stated
Speaker Count
Not stated
Annotation Type
Not stated
Status
Activederivedupdated or published within 3 years
Has Datasheet
YesderivedHugging Face card: README with a description of 300+ characters
PII Risk
Not stated
Consent Verified
Not stated
Verified By
Not stated
Date Indexed
2026-10-27
Notes / Flags
Data format: csv

Quality (DQI) (7/12)

Datasheet Fields Completed
Not stated
SPDX License ID
CC0-1.0
Redistribution Allowed?
Yes
Content Hash
6908db2a27c296158da7e69782d15df911652184derivedHugging Face card: commit sha
Submitted By
hf_data pipeline (automated)
Fetch Date
2026-10-27
Last Updated Date
2024-07-20derivedHugging Face card: lastModified
Annotation Guidelines Available?
Not stated
IAA Score
Not stated
Cleaning Notes
We cleaned the entire dataset from XML files and selected only the paragraphs.derivedHugging Face card: "We cleaned the entire dataset from XML files and selected only the paragraphs."
Benchmark Exists?
Not stated
Benchmark Citation
Not stated

Representativeness (DRI) (1/7)

Gender Distribution
Not stated
Region Coverage
Not stated
Region Distribution
Not stated
Age Range Coverage
Not stated
Register
Not stated
Marginalized Group Note
Not stated
Demographic Fields Reported Count
0

Readiness Score Breakdown

5/5
Data Coverage100%
Model Availability100%
Benchmarks100%
Documentation73%
Recency100%

Representativeness

Gender Balance
Regional Distribution
Dialect Coverage
Inclusivity

Provenance

Source URL
https://huggingface.co/datasets/cis-lmu/udhr-lid
Fetch Timestamp
27 Oct 2026
Content Hash
6908db2a27c296158da7e69782d15df911652184

Audit Log

Record indexed from Hugging Face

27 Oct 2026

Source status: Active

27 Oct 2026

Also Available In

This is the same multilingual corpus, sliced for other languages the platform indexes it under.

dataset

udhr-lid

Data format: csv

131 languagesAfar(aar)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesDangme(ada)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesAfrikaans(afr)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesAdja(ajg)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesAmharic(amh)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBambara(bam)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBamum(bax)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBariba(bba)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBaoulé(bci)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBemba(bem)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBari(bfa)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesEdo(bin)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBushi(buc)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesBulu(bum)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesChokwe(cjk)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesForro Creole(cri)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesSeychellois Creole(crs)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesDendi(ddn)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesSouthern Dagaare(dga)TextLanguage ID
HUHugging Face
Open27 Oct 2026
dataset

udhr-lid

Data format: csv

131 languagesPadang(dip)TextLanguage ID
HUHugging Face
Open27 Oct 2026

Related Datasets

dataset

WaxalNLP Multilingual Dataset (google)

Data format: parquet

8 languages15 copiesDagbani(dag)TextLM
HUHugging Face
Open27 Oct 2026
dataset

FineWeb-2 Multilingual Web Text Corpus (HuggingFaceFW)

LM · General domain · HuggingFace FineWeb Team

517 languages2 copiesDagbani(dag)TextLM
HUHugging Face
Open27 Oct 2026
dataset

FinePDFs Large-Scale Web Text Corpus (HuggingFaceFW)

Data format: parquet

511 languages2 copiesDagbani(dag)TextLM
HUHugging Face
Open27 Oct 2026