Source: Salesforce WikiText-2 raw, validation split.
File: validation.parquet
Downloaded from:
https://huggingface.co/datasets/Salesforce/wikitext/resolve/main/wikitext-2-raw-v1/validation-00000-of-00001.parquet
Dataset card: https://huggingface.co/datasets/Salesforce/wikitext

License: CC BY-SA 3.0 and GFDL, as published on that dataset card.

What the file is: English text from Wikipedia articles that Wikipedia editors
marked Good or Featured. The validation split is 60 of those articles, stored
as whitespace-separated tokens. The ingest pipeline restores hyphen,
punctuation, and contraction markers, then saves every paragraph as its own
VectorPrism chunk.

What it is not: a complete encyclopedia, a neutral sample of all subjects, or
a bias-free source. Good and Featured is an editorial filter. The articles are
English only. Nothing in this folder was rewritten to sound neutral.
