---
title: "Documentation and Metadata"
source: The Turing Way
source_url: https://github.com/the-turing-way/the-turing-way
licence: CC-BY-4.0
domain: data-and-analysis
subdomain: turing-way
date_added: 2026-04-25
---

(rr-rdm-metadata)=
# Documentation and Metadata

Having data available is of no use if it cannot be understood. Without metadata to provide provenance and context, the data can't be used effectively.
For example, a table of numbers is useless if no headings describe what the columns/rows contain.
Therefore you should ensure that open datasets include consistent metadata, that is information about the data so that the data is fully described.
This requires that information accompanying data is captured in documentation and metadata. 

(rr-rdm-metadata-documentation)=
## Documentation
Documentation provides context for your work. 
It allows your collaborators, colleagues and future you to understand what has been done and why.

Data documentation can be done on different levels. 
All documentation accompanying data should be written in clear, plain language. 
Documentation allows data users have sufficient information to understand the source, strengths, weaknesses, and analytical limitations of the data so that they can make informed decisions when using it. 

```{figure} ../../../figures/documentation.*
---
name: documentation
alt: The figure goes through a dark wood setting lights along the way. The lights are blocks of text - one can see that these are pieces of documentation. They make it easy for colleagues to find their way. In the darkness one sees another figure - someone got lost in the woods where no documentation was available.
---
Illustration about peer review.
_The Turing Way_ project illustration by Scriberia. Used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807.
```

(rr-rdm-readme)=
### README for datasets

A README file provides information about data (or software) and is intended to help ensure that the data can be correctly interpreted and used, by yourself or others.
A README file is generally shared together with an open dataset, or can be the [landing page of your software/project repository](#pd-project-repo-readme).

README text files should describe the methods used for data collection and analysis and include data/software-specific information (parameters, variables, column headings, symbols used).
See [Make a README](https://www.makeareadme.com/) for more information on why README files are important and how you can set up your own README files.

#### README tips and examples

* Create one README file for each dataset
* Name the file README
* Use plain text file (`README.txt` or `README.md`).
  Or use README.pdf when text formatting is important for your file.

Example templates:

* [Cornell Template](https://cornell.app.box.com/v/ReadmeTemplate)
* [4TU.ResearchData’s guidelines for readme files](https://data.4tu.nl/s/documents/Guidelines_for_creating_a_README_file.pdf)

(rr-rdm-codebook)=
### Codebook or Data dictionary

A data dictionary or codebook ensures that the dataset is interpretable and reusable in the future, and has also been called the '[single most important piece of documentation you will create in your research study](https://cghlewis.com/blog/data_dictionary/)'. 
A guiding question when setting up your data dictionary is: Can another researcher analyse/interpret the data using solely the codebook?

* Useful codebooks are human and machine readable (information provided can be accessed using automated approaches) and consistent.
* The codebook should at least list all variables and labels that are included in the dataset/article, and may include additional variables. 
* You should explain the (range) of numerical values, units of measurement, sources, classification schemes used, or labels of questions asked.
* You can also include documentation or metadata not shown in the dataset itself, with an explanation of why it is not included in the final data/article, or outline in detail what missing data means (if there is any in the dataset).

You can check the [OSF guidance for data dictionaries](https://help.osf.io/article/217-how-to-make-a-data-dictionary) for more detailed variable guidance. 

When using a spreadsheet as a codebook, the same formatting principles apply as to regular [spreadsheet data](#rr-rdm-spreadsheets) 

#### Data dictionary examples

* See Tables 1 and 2 in [Horstmann et al. 2020](https://doi.org/10.1027/1015-5759/a000620) or their [templates in spreadsheet format](https://osf.io/nerpa/)


(rr-rdm-metadata-metadata)=
## Metadata

Metadata is information about the data, descriptors that facilitate cataloguing data and data discovery. 
Often, metadata are intended for machine reading.

When data is submitted to a trusted data repository, the machine-readable metadata is generated by the repository. 
If the data is not in a repository a text file with machine-readable metadata can be added as part of the documentation.

- The type of research and the nature of the data also influence what kind of documentation is necessary. 
- The level of documentation and metadata [{term}`def<Metadata>`] will vary according to the project, and the range of people the data needs to be understood by.
- Examples of documentation may include items like [data dictionaries](https://help.osf.io/hc/en-us/articles/360019739054-How-to-Make-a-Data-Dictionary) (see [here for a template](https://data.nal.usda.gov/data-dictionary-blank-template)) or codebooks, protocols, logbooks or lab journals, README files, research logs, analysis syntax, algorithms and code comments.  
- Variables should be defined and explained using data dictionaries or codebooks.
- Data should be stored in logical and hierarchical folder structures, with a README file used to describe the structure.
The README file is helpful for others and will also help you find your data in the future {cite:ps}`Fuchs2018documentation`.
See the [README template from Cornell](https://cornell.app.box.com/v/ReadmeTemplate) for an example.
- It is best practice to use recognised community metadata standards to make it easier for datasets to be combined.


(rr-rdm-metadata-standards)=
### Community Standards - Metadata
The use of community-defined standards for metadata is vital for reproducible research and allows for the comparison of heterogeneous data from multiple sources, domains and disciplines.
Metadata standards are also discipline-specific.
For example, for brain data, the [Brain Imaging Data Structure](https://doi.org/10.25504/FAIRsharing.rd1j6t) is the standard to use.
Not every discipline may use metadata standards, however.
You can see if your discipline uses metadata standards through [FAIRsharing](https://fairsharing.org/), a resource to identify and cite the metadata or identifier schemas, databases or repositories that exist for your data and discipline.

In this case, a text file with discipline specific metadata can be added as part of the documentation.

Want to learn more about Metadata and Metadata Standards? Watch an [introduction video](https://commons.esipfed.org/node/1422).


(rr-rdm-metadata-pid)=
### PID Metadata

{abbr}`PID (Persistent Identifier)` metadata is metadata submitted to PID registration agencies, most repositories handle PID metadata automatically through API integration, it enables your work to be discoverable across disciplines and platforms.

PID metadata includes core properties like:
- Creator names and affiliations
- Title and description
- Publication date and resource type
- Relationships to other research outputs (such as citations or dataset-paper links)

Unlike domain-specific metadata (such as the Brain Imaging Data Structure for neuroimaging data), PID metadata follows standardized schemas that are discipline agnostic.
Both types of metadata are valuable - PID metadata enables broad discoverability while domain-specific metadata captures the detailed context needed for reuse within your field.

Learn about how repositories generate PID metadata and the distinction between these metadata types in our [chapter on persistent identifiers](#rr-rdm-pid-metadata).


(rr-rdm-metadata-tagging)=
## Tagging

Tags are keywords assigned to files, and a way to add metadata to a file to organise them more flexibly.
While a file can only be in one folder at a time, it can have an unlimited number of tags. 

Some tips include:
- Use short tag names (one or two words)
- Be consistent with tags
- Not all file formats allow tags, and when files are transferred tags may be stripped

See [Tagging and Finding Your Files by MIT libraries](https://libguides.mit.edu/metadataTools)) for more information. 

(rr-rdm-metadata-resources)=
## Additional Resources
- Videos on [Data Description](https://www.youtube.com/watch?v=sg3P_V8PIes) & [Documentation and Data Quality](https://www.youtube.com/watch?v=3ByfQWDcavg) from the [TU Delft Open Science MOOC](https://online-learning.tudelft.nl/courses/open-science-sharing-your-research-with-the-world/).
- Example of data documentation by {cite:ps}`Larsen2021Documentation`
- [Webinar: The Data You Document are the Data We Love](https://youtu.be/SoFxBN-Jnbg?t=1133)
- [Slides: FAIRify your data: data documentation and metadata](https://osf.io/wbr7t/)
- [Controlled vocabularies for the social sciences: what they are, and why we need them](https://odissei-data.nl/en/2022/10/controlled-vocabularies-for-the-social-sciences-what-they-are-and-why-we-need-them/)
- [Research Data Management: Metadata](https://libguides.ucd.ie/data/metadata)
- Data dictionaries and codebooks by {cite:ps}`Buchanan2021dictionaries`.
