opendata-rfb

Overview

opendata-rfb reads the Receita Federal do Brasil (RFB) CNPJ open data (the company registry) into typed pandas DataFrames. It covers every dataset the RFB publishes:

Data source and cadence. The data comes from the RFB's public share (arquivos.receitafederal.gov.br), which publishes one folder per reference month (YYYY-MM), refreshed monthly. A reader takes the month as its argument; a month the share does not publish raises an error rather than falling back to another one.

⚠️ Size. The archives are large and are never held in memory: each reader streams every ZIP to disk and yields typed chunks. Estabelecimentos is the heaviest (ten parts, up to 2.3 GB each compressed) and Simples.zip inflates to ~3.2 GB. Plan for disk space and a long first run; pass path_workdir= to keep the ZIPs so a rerun resumes instead of downloading again.

Documentation

The full documentation is published at https://guilhermegor.github.io/opendata-rfb/:

• Usage: install, readers, basics
• Examples: task-oriented recipes
• API Reference: every public function
• FAQ: common questions and troubleshooting
• Changelog: release history

To serve it locally: poetry run poe docs_serve.

Getting Started

Python 3.10+ is the only prerequisite.

Install

pip install opendata-rfb
pip install 'opendata-rfb[parquet]'   # optional: Parquet export (pulls in pyarrow)

Or with Poetry:

poetry add opendata-rfb
poetry add 'opendata-rfb[parquet]'   # optional: Parquet export (pulls in pyarrow)

To work on the library itself, see CONTRIBUTING.md.

Usage

Every dataset reader (read_companies, read_establishments, read_partners, read_simples) takes a reference month (YYYY-MM) and yields chunks of int_chunk_rows rows (default 500,000), so memory stays flat. read_lookup is the exception: it takes str_table, path_zip, str_url, str_etag and returns a single DataFrame, since each lookup table is small. Every row carries provenance columns (url, updated_at, source_key, package_version, ingestion_run_id, content_hash, source_etag).

Empresas. capital_social is an exact Decimal, never a float.

from opendata_rfb import read_companies

for df_chunk in read_companies("2026-10"):
    ...

Estabelecimentos. Adds the full cnpj_completo and its validity flag cnpj_valido (alphanumeric-aware).

from opendata_rfb import read_establishments

for df_chunk in read_establishments("2026-10"):
    ...

Socios. Masked CPFs (***240659**) are kept verbatim, never unmasked; ***000000** as representative document becomes null.

from opendata_rfb import read_partners

for df_chunk in read_partners("2026-10"):
    ...

Simples. Booleans and real dates, with the RFB's 00000000 read as null.

from opendata_rfb import read_simples

for df_chunk in read_simples("2026-10"):
    ...

Lookups. read_lookup reads one downloaded lookup ZIP (one of CNPJ_LOOKUP_NAMES).

from pathlib import Path

from opendata_rfb import read_lookup
from opendata_rfb._internal.utils.stream_downloader import download_file_streaming
from opendata_rfb.ingestion.cnpj._share import file_url, latest_reference_month

str_url = file_url(latest_reference_month(), "Cnaes.zip")
path_zip, str_etag = download_file_streaming(str_url, Path("data/raw/Cnaes.zip"))
df_cnaes = read_lookup("Cnaes", path_zip, str_url, str_etag)

Parquet. export_companies_parquet("2026-10", path) writes Empresas to one Parquet file for lazy querying (DuckDB, Polars scan_parquet); it needs the parquet extra.

⚠️ Do not read these files with Polars directly: it silently misparses the .zip and corrupts latin-1 text.

Project Structure

opendata-rfb/
├── .github/              workflows, CODEOWNERS, PR template
├── assets/               logo
├── bin/                  gate scripts and helpers
├── docs/                 published documentation (mkdocs)
├── src/opendata_rfb/
│   ├── __init__.py       the public API (__all__)
│   ├── ingestion/cnpj/   one reader per RFB dataset
│   └── _internal/        private: contracts, utils, ports
├── tests/
│   ├── unit/
│   └── integration/
├── pyproject.toml
└── poe_tasks.toml        task runner entry points

Development

git clone https://github.com/guilhermegor/opendata-rfb.git
cd opendata-rfb
bash bin/venv.sh                           # first time only: creates .venv; poe exists only after this
poetry run poe init                        # venv, git hooks, repo rules, security, Pages
poetry run poe enable_pages                # maintainers, one time, after the first PyPI release

If bin/venv.sh just installed Poetry into ~/.local/bin, add that directory to PATH before poetry run works. init also runs GitHub helpers that only maintainers can use; on a fork they warn and continue. enable_pages is already done for this repo.

Daily commands:

poetry run poe unit_tests                  # unit suite
poetry run poe integration_tests           # integration suite
poetry run poe lint                        # ruff, codespell and the structure gates
poetry run poe docs_serve                  # serve the docs locally

Public API

The public surface is whatever the package __init__.py lists in __all__; everything under _internal/ is private.

• read_companies(str_month, int_chunk_rows=500_000, path_workdir=None): Empresas chunks.
• read_establishments(...): Estabelecimentos chunks, same signature.
• read_partners(...): Socios chunks, same signature.
• read_simples(...): Simples chunks, same signature.
• export_companies_parquet(str_month, path_dest, int_chunk_rows=500_000, path_workdir=None): Empresas to one Parquet file (needs the parquet extra).
• read_lookup(str_table, path_zip, str_url, str_etag): one of the six lookup tables (CNPJ_LOOKUP_NAMES) as a typed DataFrame.
• CNPJ_LOOKUP_NAMES, __version__.

Publishing

Releases go out from the release workflows under .github/workflows/ through PyPI trusted publishing (OIDC); the git tag is the version.

Contributing

See CONTRIBUTING.md for the branch, commit and review rules.

License

This project is licensed under MIT; see LICENSE.

Authors

• guilhermegor — GitHub | LinkedIn

Useful Links

• Documentation
• PyPI
• GitHub Repository
• Issue Tracker

_{Generated from the lib-minimal template via BlueprintX. Unofficial, community library. Not affiliated with or endorsed by the Receita Federal do Brasil; the emblem is theirs.}