Metadata-Version: 2.5
Name: us-gov-data-dictionary
Version: 0.1.0
Summary: Machine-readable data dictionaries for U.S. government datasets (SEC N-PORT, 13F, Forms 3/4/5, Form D, N-MFP, Reg A, N-CEN): official field definitions with citations, code lists and observed values.
Project-URL: Homepage, https://alexbocio.github.io/us-gov-data-dictionary/
Project-URL: Source, https://github.com/AlexBocio/us-gov-data-dictionary
Project-URL: Issues, https://github.com/AlexBocio/us-gov-data-dictionary/issues
Author: Alex Bocio (Alt Data Forge)
License-Expression: Apache-2.0 AND CC-BY-4.0
License-File: LICENSE
License-File: LICENSE-CODE
Keywords: 13F,CSVW,EDGAR,Form 4,Form D,ISO 11179,N-CEN,N-MFP,N-PORT,Regulation A,SEC,SKOS,codebook,data dictionary,metadata
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Office/Business :: Financial
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.9
Requires-Dist: pyyaml>=6
Description-Content-Type: text/markdown

# U.S. Government Data Dictionary

**Machine-readable data dictionaries for U.S. government datasets** — every field and every code, with its
official definition and a citation to the agency document it came from, and the values actually found in the
published files.

> Status: **seven SEC dictionaries, 1,469 fields** - more agencies being added.

| Dictionary | Fields | Periods covered | Layers |
|---|---|---|---|
| [Form N-PORT](docs/sec_nport.html) (fund holdings) | 277 | 2019 Q4 - 2026 Q2 | official, code lists, observed values |
| [Form 13F](docs/sec_13f.html) (institutional holdings) | 68 | 2013 Q2 - Aug 2026 | official |
| [Forms 3, 4, 5](docs/sec_form345.html) (insider ownership) | 147 | 2006 Q1 - 2026 Q2 | official |
| [Form D](docs/sec_form_d.html) (exempt offerings) | 108 | 2008 Q1 - 2026 Q2 | official |
| [Form N-MFP](docs/sec_nmfp.html) (money market funds) | 189 | 2010 Q4 - 2022 Q2 | official |
| [Regulation A](docs/sec_rega.html) (small offerings) | 175 | 2015 Q2 - 2026 Q2 | official |
| [Form N-CEN](docs/sec_ncen.html) (fund census) | 505 | 2026 Q1 | official |

Files: `dictionaries/<source>.yaml` (all layers) · `csvw/<source>-metadata.json` (W3C CSVW) ·
`skos/<source>-codelists.jsonld` (W3C SKOS code lists) · `docs/` (web pages with schema.org Dataset markup).

**Found while building** (recorded in each dictionary's `known_caveats`): the SEC's Form D field list is not
valid JSON and garbles two field names; Regulation A data files misspell their own columns (`PUBLICYTRADED`,
`IFUNREGSITEREDNONE`); N-MFP's field list names columns its files spell differently; the N-PORT field list is
re-shipped unchanged with every historical quarter, so a field "defined" there may not exist in older files.

## Install (Python)

```
pip install us-gov-data-dictionary
```

```python
import us_gov_data_dictionary as dd
dd.sources()                                         # the seven SEC dictionaries
dd.field("sec_nport", "IDENTIFIER_TICKER")           # official definition, form item, observed values
dd.search("ticker")                                  # every field whose name or definition mentions it
dd.code_list("sec_nport", "NEW_FILE_NUM_TYPE")       # an official code list as {code: meaning}
```

Command line: `us-gov-dd list`, `us-gov-dd show sec_nport IDENTIFIER_TICKER`, `us-gov-dd search ticker`,
`us-gov-dd codes sec_nport`. The package bundles the YAML files in `dictionaries/`; its only dependency is PyYAML.

## Why this exists

Agencies publish their data with field lists, but the meaning of a field often lives somewhere else: in the form
instructions, in an XML schema, or in a stylesheet. Code lists are frequently missing from the data documentation
itself, and a field's real content can differ from what its one-word label suggests.

Example — SEC Form N-PORT: the data-set documentation describes `IDENTIFIER_TICKER` as just "Ticker". The form
instructions (Item C.1.e.ii) say a ticker is reported only *"if ISIN is not available"*, with no requirement that it
be an exchange symbol. In the published filings the field holds stock tickers on common-stock lines, the issuer's
ticker on bond lines, agency pool codes (FN, GN, FG) on mortgage-backed lines, and futures or currency codes on
derivative lines. Read as a stock ticker everywhere, it links mortgage pools to unrelated listed companies.

## What each dictionary contains

Each source is one YAML file. Every field carries up to three layers, kept separate so a reader always knows who is speaking:

| Layer | Content | Source |
|---|---|---|
| `official` | the agency's definition, verbatim, with form item number, type and length | agency data-set documentation and form instructions |
| `codes` | the official list of allowed codes and their meanings, joined to a field only when the filed values confirm it | agency technical specifications (e.g. EDGAR XML schemas) |
| `observed` | what the published files actually contain: fill rate, distinct values, value shapes, placeholder values | measured across every published period |

Every dictionary lists the official documents it was built from, with URL, version and SHA-256, and records which
periods each field actually appears in (read from the data files, not from the field list).

## Identifiers

Company identifiers referenced here are open identifiers only: **SEC CIK**, **LEI** (GLEIF) and **FIGI** (OpenFIGI). CUSIP and
CUSIP-derived ISIN values are not published in this repository.

## Licence

Agency text quoted here is a U.S. federal government work and in the public domain. The structure and measurements in
this repository are licensed under CC BY 4.0. The Python code in `src/` is licensed under Apache-2.0 (`LICENSE-CODE`).
