U.S. Government Data Dictionary

Machine-readable data dictionaries for U.S. government datasets — every field and every code, with its official definition and a citation to the agency document it came from, and the values actually found in the published files.

Status: seven SEC dictionaries, 1,469 fields - more agencies being added.

Files: dictionaries/<source>.yaml (all layers) · csvw/<source>-metadata.json (W3C CSVW) · skos/<source>-codelists.jsonld (W3C SKOS code lists) · docs/ (web pages with schema.org Dataset markup).

Found while building (recorded in each dictionary's known_caveats): the SEC's Form D field list is not valid JSON and garbles two field names; Regulation A data files misspell their own columns (PUBLICYTRADED, IFUNREGSITEREDNONE); N-MFP's field list names columns its files spell differently; the N-PORT field list is re-shipped unchanged with every historical quarter, so a field "defined" there may not exist in older files.

Install (Python)

pip install us-gov-data-dictionary

import us_gov_data_dictionary as dd
dd.sources()                                         # the seven SEC dictionaries
dd.field("sec_nport", "IDENTIFIER_TICKER")           # official definition, form item, observed values
dd.search("ticker")                                  # every field whose name or definition mentions it
dd.code_list("sec_nport", "NEW_FILE_NUM_TYPE")       # an official code list as {code: meaning}

Command line: us-gov-dd list, us-gov-dd show sec_nport IDENTIFIER_TICKER, us-gov-dd search ticker, us-gov-dd codes sec_nport. The package bundles the YAML files in dictionaries/; its only dependency is PyYAML.

Why this exists

Agencies publish their data with field lists, but the meaning of a field often lives somewhere else: in the form instructions, in an XML schema, or in a stylesheet. Code lists are frequently missing from the data documentation itself, and a field's real content can differ from what its one-word label suggests.

Example — SEC Form N-PORT: the data-set documentation describes IDENTIFIER_TICKER as just "Ticker". The form instructions (Item C.1.e.ii) say a ticker is reported only "if ISIN is not available", with no requirement that it be an exchange symbol. In the published filings the field holds stock tickers on common-stock lines, the issuer's ticker on bond lines, agency pool codes (FN, GN, FG) on mortgage-backed lines, and futures or currency codes on derivative lines. Read as a stock ticker everywhere, it links mortgage pools to unrelated listed companies.

What each dictionary contains

Each source is one YAML file. Every field carries up to three layers, kept separate so a reader always knows who is speaking:

Every dictionary lists the official documents it was built from, with URL, version and SHA-256, and records which periods each field actually appears in (read from the data files, not from the field list).

Identifiers

Company identifiers referenced here are open identifiers only: SEC CIK, LEI (GLEIF) and FIGI (OpenFIGI). CUSIP and CUSIP-derived ISIN values are not published in this repository.

Licence

Agency text quoted here is a U.S. federal government work and in the public domain. The structure and measurements in this repository are licensed under CC BY 4.0. The Python code in src/ is licensed under Apache-2.0 (LICENSE-CODE).