DATASETHugging Face

5/5

fineweb-2

Andoni(ann)Nlptext-generationWest Africa

View Source

Dataset Info

Language

Andoni (ann)

Modality

Nlp

Domain

NLP

Size

n>1T n (samples)

Institution

caoyuji

Last Updated

27 Oct 2026

License & Access

SPDX identifierodc-by

Licensed under odc-by. Review the source terms before use.

RedistributionYes

PII RISK

Unknown risk

Full Record

29 of 46 fields stated

Every column the registry tracks. Values marked derivedwere filled by a rule or read from the source's text rather than stated in its metadata; the rule or the quoted sentence is shown under the value.

Registry columns (20/27)

iso code: ann
Dataset name: fineweb-2
Modality: Nlp
Type: dataset
Link: https://huggingface.co/datasets/caoyuji/fineweb-2
NLP Task: text-generation
Domain: NLP
Data Format: csv, parquetderivedHugging Face card: file formats
Size Metric: n (samples)
Size Value: n>1T n (samples)
License: odc-by
Open Access: Open
Source Repository: Hugging Face
Year Published: 2025
Creator/Author(s): caoyuji
Funder/Partner: Google
Dialect Coverage: Not stated
Gender Balance: Not stated
Speaker Count: Not stated
Annotation Type: Automatic
Status: Activederivedupdated or published within 3 years
Has Datasheet: YesderivedHugging Face card: README with a description of 300+ characters
PII Risk: Not stated
Consent Verified: Not stated
Verified By: Not stated
Date Indexed: 2026-10-27
Notes / Flags: Not stated

Quality (DQI) (7/12)

Datasheet Fields Completed: Not stated
SPDX License ID: ODC-BY-1.0
Redistribution Allowed?: Yes
Content Hash: 4057e43d8bcc1ed997df3bc04c24de9cfc38c3bfderivedHugging Face card: commit sha
Submitted By: hf_data pipeline (automated)
Fetch Date: 2026-10-27
Last Updated Date: 2025-12-08derivedHugging Face card: lastModified
Annotation Guidelines Available?: Not stated
IAA Score: Not stated
Cleaning Notes: We mostly kept the original 🍷 FineWeb set of filters, and do not create new filters targeting individual languages. As such, we had to extensively ablate on di…derivedHugging Face card: "### Data Filtering 🧹 We mostly kept the original 🍷 FineWeb set of filters, and do not create new filters targeting individual languages."
Benchmark Exists?: Not stated
Benchmark Citation: Not stated

Representativeness (DRI) (2/7)

Gender Distribution: Not stated
Region Coverage: Not stated
Region Distribution: Not stated
Age Range Coverage: Not stated
Register: Written
Marginalized Group Note: Not stated
Demographic Fields Reported Count: 0

Readiness Score Breakdown

5/5

Data Coverage95%

Model Availability100%

Benchmarks100%

Documentation83%

Recency100%

Representativeness

Gender Balance

Regional Distribution

Dialect Coverage

Inclusivity

Provenance

Source URL

https://huggingface.co/datasets/caoyuji/fineweb-2

Fetch Timestamp

27 Oct 2026

Content Hash

4057e43d8bcc1ed997df3bc04c24de9cfc38c3bf

Audit Log

Record indexed from Hugging Face

27 Oct 2026

Source status: Active

27 Oct 2026

Other Copies

This is a copy of the resource below, which the platform counts and shows in search.

dataset

fineweb-2

text-generation · NLP domain · HuggingFaceFW

517 languages2 copiesAndoni(ann)Tabulartext-generation

HUHugging Face

Open27 Oct 2026

Also Available In

This is the same multilingual corpus, sliced for other languages the platform indexes it under.

dataset

fineweb-2

text-generation · NLP domain · HuggingFaceFW

517 languages2 copiesAbé(aba)Tabulartext-generation

HUHugging Face

Open27 Oct 2026

dataset

FineWeb-2 Multilingual Web Text Corpus (HuggingFaceFW)

Large-scale multilingual web text corpus tagged for Abidji (abi). 13 DS returned: 8 included, 5 excluded (KefranAbg/finepdfs fork of finepdfs; caoyuji/finewe…

517 languages2 copiesAbidji(abi)TextLM

HUHugging Face

Open24 Jul 2026

dataset