worldbank-commodities 0.8.1

Python wrapper to extract World Bank Pink Sheet commodity price data (oil, gas, metals, agriculture).

worldbank-commodities

A small, dependency-light Python wrapper around the World Bank "Pink Sheet" (Commodity Markets) price data.

The World Bank publishes monthly and annual commodity price series (crude oil, natural gas, coal, metals, fertilizers, and many agricultural products) as Excel workbooks whose download URLs carry a hash that changes every month. This wrapper handles that for you:

• Auto-discovers the current Monthly/Annual workbook URLs from the official Commodity Markets page, so there are no hardcoded links that break every month.
• Parses the awkward multi-row-header sheets into tidy pandas DataFrames (long or wide), detecting the units row automatically.
• Caches the raw workbook on disk (24 h TTL) so repeat calls are fast.
• Exposes commodity metadata (name and unit) and filters by commodity name and date range.
• Cleans the World Bank's "not available" tokens (…, ..) to NaN.

No API key required; the data is public.

Coverage

71 monthly series back to 1960, updated monthly. Groups include:

Group	Examples
Energy	Crude oil (average/Brent/Dubai/WTI), coal, natural gas (US/Europe/LNG Japan)
Metals & precious	Aluminum, copper, iron ore, nickel, zinc, tin, lead, gold, silver, platinum
Fertilizers	Phosphate rock, DAP, TSP, urea, potassium chloride
Agriculture	Grains, vegetable oils & oilseeds, cocoa/coffee/tea, meat, sugar, cotton, rubber, timber

Series flagged ** by the World Bank have methodology/source breaks; see the Description sheet in the source workbook.

Quickstart

New to Python tooling? These four steps take you from nothing to a spreadsheet of commodity prices. You need git and uv (a fast Python package manager) installed.

Install uv if you do not have it:

# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Then:

# 1. Get the code
git clone https://github.com/hmorao95/worldbank-commodities.git
cd worldbank-commodities

# 2. Install it (uv creates a .venv and pulls dependencies; no manual Python setup)
uv sync

# 3. See what commodities are available
uv run worldbank-commodities list-commodities

# 4. Save every monthly price series to a CSV you can open in Excel
uv run worldbank-commodities to-csv prices.csv --freq monthly

A bare output filename is written into the repo's outputs/ folder automatically (so prices.csv becomes outputs/prices.csv). That folder's contents are gitignored, so extracts stay out of version control. Pass a path with a directory (e.g. data/prices.csv or an absolute path) to write elsewhere.

From there, the same commands take options to narrow or reshape the data:

# Save to Excel instead of CSV
uv run worldbank-commodities to-excel prices.xlsx --freq monthly

# Just one commodity (case-insensitive, matches on any part of the name)
uv run worldbank-commodities to-csv brent.csv --freq monthly --commodities "Crude oil, Brent"

# A date range, one column per commodity (wide)
uv run worldbank-commodities to-csv oil.csv --freq monthly \
    --commodities "crude oil" --start 2000-01 --end 2024-12 --wide

# Annual, real (inflation-adjusted) prices instead of nominal monthly
uv run worldbank-commodities to-csv real.csv --freq annual_real

# Keep an existing file current: add only new and revised rows
uv run worldbank-commodities update-csv prices.csv --freq monthly

# See every command and flag
uv run worldbank-commodities --help

See Reference for the full list of commands and options.

That is it. outputs/prices.csv now holds one row per observation (date, mdates, commodity, price, unit). mdates is a Stata-style monthly id such as 1960m1, handy for merging with Stata time series; it is added only for the monthly frequency (in both long and wide output).

Note: the worldbank-commodities command lives inside the project's .venv, which is why each command starts with uv run. If you skip uv run and get "command not found", that is the reason. To get a global command, see Command line below.

To install the library into your own Python environment instead, use pip:

pip install worldbank-commodities

Requires Python 3.10 or newer (uv installs a suitable Python for you if needed).

Usage from Python

Once installed you can call the library from your own scripts. Run these with uv run python your_script.py (or uv run python for an interactive session).

get_prices() returns a pandas DataFrame, the standard table type for data work in Python.

from worldbank_commodities import WorldBankCommodities

wb = WorldBankCommodities()

# All 71 commodities + their units
wb.list_commodities()

# Tidy long DataFrame: date, mdates, commodity, price, unit
wb.get_prices(freq="monthly")

# Just crude oil + European gas since 2000, wide (one column per commodity)
wb.get_prices(
    freq="monthly",
    commodities=["Crude oil, Brent", "Natural gas, Europe"],
    start="2000-01",
    wide=True,
)

# Annual nominal or real (deflated) prices
wb.get_prices(freq="annual")  # nominal
wb.get_prices(freq="annual_real")  # real

# One-liner to CSV or Excel
wb.to_csv("brent.csv", freq="monthly", commodities=["Crude oil, Brent"], wide=True)
wb.to_excel("brent.xlsx", freq="monthly", commodities=["Crude oil, Brent"], wide=True)

# Incremental update: only append observations newer than what's already saved.
# First run writes the full history; later runs just tack on the new month(s).
wb.update_csv("commodities_monthly.csv", freq="monthly")

What the options mean:

• freq picks the dataset: "monthly", "annual" (nominal prices), or "annual_real" (inflation-adjusted prices).
• commodities filters by name. Matching is case-insensitive and matches on any part of the name, so commodities=["crude oil"] returns all four crude-oil series. Leave it out to get everything.
• start and end limit the date range, for example start="2000-01".
• wide=False (the default) gives one row per observation, which is easy to filter and plot. wide=True gives one column per commodity, which is handy for a side-by-side spreadsheet. Run wb.list_commodities() to see the exact names.

Command line

The CLI is generated from the library with python-fire: each method becomes a subcommand and each parameter becomes a flag, so there is no separate set of options to learn. Run worldbank-commodities --help to see them all.

Inside a uv project the console script lives in .venv, so call it with uv run (or activate the venv first). python -m worldbank_commodities works too:

uv run worldbank-commodities list-commodities

To make the command available everywhere (outside this project, on your PATH), install it as a uv tool once:

uv tool install .
worldbank-commodities list-commodities

The examples below use the bare command; prefix them with uv run if you have not installed the tool globally.

# List every commodity and its unit
worldbank-commodities list-commodities

# Print a table to the terminal
worldbank-commodities get-prices --freq monthly

# Extract to a CSV
worldbank-commodities to-csv prices.csv --freq monthly

# Extract to an Excel workbook
worldbank-commodities to-excel prices.xlsx --freq monthly

# Filtered by one commodity (quote the name so the shell keeps it together)
worldbank-commodities get-prices --freq monthly \
    --commodities "Crude oil, Brent" --start 2010-01 --wide --out brent.csv

# Several commodities: pass a list. On Windows cmd wrap it in double quotes
# and use single quotes inside: --commodities "['Crude oil, Brent', 'Gold']"
worldbank-commodities get-prices --freq monthly \
    --commodities "['Crude oil, Brent', 'Natural gas, Europe']" --wide

# Incremental: add new and revised rows to an existing long CSV
worldbank-commodities update-csv commodities_monthly.csv --freq monthly

# Show what is missing from a CSV without writing it
worldbank-commodities new-observations commodities_monthly.csv --freq monthly

# Aggregate price indices (Energy, Agriculture, Metals, ...; base 2010=100)
worldbank-commodities get-indices --freq monthly --indices energy

# Full series definitions from the workbook's Description sheet
worldbank-commodities describe

Reference

Run worldbank-commodities --help, or worldbank-commodities <command> --help for a single command, to see this from the CLI.

CLI commands

Command	Positional	Options
list-commodities	(none)	--freq
list-indices	(none)	--freq
describe	(none)	--freq
get-prices	(none)	--freq --commodities --start --end --wide --change --out
get-indices	(none)	--freq --indices --start --end --wide --change --out
to-csv	PATH	--freq --commodities --start --end --wide --change
to-excel	PATH	--freq --commodities --start --end --wide --change
to-parquet	PATH	--freq --commodities --start --end --wide --change
to-json	PATH	--freq --commodities --start --end --wide --change
update-csv	PATH	--freq --commodities --start --end
new-observations	EXISTING	--freq --commodities --start --end --include-revisions

For get-prices, --out picks the format by extension (.xlsx/.xls Excel, .parquet Parquet, .json JSON, otherwise CSV); without --out it prints a table. A bare output filename (no directory) is written into the outputs/ folder; pass a path with a directory to write elsewhere.

Options

Option	Values	Meaning
--freq	monthly (default), annual, annual_real	Dataset: nominal monthly, nominal annual, or real (deflated) annual.
--commodities	name or list of names	Case-insensitive substring filter. Omit for all. A single name is a plain string; several use a list (see Windows quoting above).
--start, --end	e.g. 2000-01, 2000	Inclusive date bounds.
--wide	flag	One column per commodity indexed by date (plus a leading mdates column for monthly). Default is long: one row per observation.
--change	mom, yoy	Return percentage changes instead of levels: month-over-month, or year-over-year (12 months for monthly, 1 step for annual).
--include-revisions	flag (default on)	For new-observations: also return values the World Bank has since revised, not only unseen rows.

Python API

WorldBankCommodities(cache_dir=None, cache_ttl_hours=24.0, timeout=60.0, session=None)

Method	Returns
list_commodities(freq="monthly")	DataFrame of commodity, unit
list_indices(freq="monthly")	DataFrame of index names
describe(freq="monthly")	Definitions: group, commodity, description
get_prices(freq="monthly", commodities=None, start=None, end=None, wide=False, change=None)	Long or wide DataFrame
get_indices(freq="monthly", indices=None, start=None, end=None, wide=False, change=None)	Aggregate price indices (base 2010=100)
to_csv(path, **kwargs)	Writes CSV; returns the Path
to_excel(path, **kwargs)	Writes .xlsx; returns the Path
to_parquet(path, **kwargs)	Writes .parquet; returns the Path
to_json(path, **kwargs)	Writes JSON records; returns the Path
update_csv(path, freq="monthly", commodities=None, start=None, end=None)	Upserts the CSV; returns the added/revised rows
new_observations(existing, freq="monthly", commodities=None, start=None, end=None, include_revisions=True)	Rows missing from existing (DataFrame or CSV path)

The to_* writers forward **kwargs to get_prices (freq, commodities, start, end, wide, change). The long output columns are [date, commodity, price, unit], plus mdates for the monthly frequency. With change="mom" or "yoy" the values are percentage changes and the unit is %.

Incremental updates

update_csv() (CLI: update-csv) keeps a long-format CSV current without rewriting the whole file each run:

• First run (file missing) writes the full extract.
• Later runs read what's already saved and upsert on (commodity, date): they append new observations and overwrite any values the World Bank has since revised. That covers newer months for commodities already present and commodities that were not extracted before.
• If nothing changed, the file is left untouched and an empty frame is returned.

new_rows = wb.update_csv("commodities_monthly.csv", freq="monthly")
print(f"added {len(new_rows)} observations")

This works on the tidy long format ([date, commodity, price, unit]); wide CSVs are not supported for incremental merging.

Just the delta, in memory

If you keep your data somewhere other than a CSV, new_observations() is the file-free building block. Give it what you already have (a long DataFrame or a long-CSV path) and it returns only the rows you are missing: newer dates for commodities you track, plus every row of commodities you have never extracted.

have = my_store.load()  # any long DataFrame with [date, commodity, price]
missing = wb.new_observations(have, freq="monthly")
my_store.append(missing)

# Strictly-new rows only (ignore World Bank back-revisions):
missing = wb.new_observations(have, freq="monthly", include_revisions=False)

It handles the cases you would expect:

full = wb.get_prices("monthly")

# Already have everything -> nothing to fetch
wb.new_observations(full, freq="monthly")  # empty

# Missing the latest months -> only those rows come back
wb.new_observations(full.iloc[:-100], freq="monthly")

# A commodity never extracted -> its whole history is returned
wb.new_observations(full[full["commodity"] != "Gold"], freq="monthly")

# Accepts a CSV path too, not just a DataFrame
wb.new_observations("commodities_monthly.csv", freq="monthly")

Development

Managed with uv. The package lives under a src/ layout (src/worldbank_commodities/) and tests under tests/.

uv sync                     # create .venv and install deps (dev + typing groups)
uv run ruff check .         # lint
uv run ruff format .        # format
uv run mypy                 # strict type checking
uv run pytest               # tests + coverage (network-mocked, no downloads)

Full quality gate (mirrors CI):

uv run ruff check . && uv run ruff format --check .
uv run codespell src tests README.md CHANGELOG.md
uv run deptry src
uv run interrogate -c pyproject.toml src tests   # docstring coverage >= 95%
uv run mypy
uv run pytest

pre-commit runs ruff, codespell, and mypy on every commit:

uv run pre-commit install       # one-time, enables the git hook
uv run pre-commit run --all-files

CI (.github/workflows/ci.yml) runs the same gate on every push and pull request.

Build and publish

Build the sdist and wheel with uv:

uv build

Publishing a GitHub release triggers .github/workflows/publish.yml, which runs uv build and uv publish to PyPI using Trusted Publishing, so no API token is stored in the repo. To publish by hand instead, run uv publish with your own credentials.

Changelog

See CHANGELOG.md (Keep a Changelog format) or the GitHub releases for per-version notes.

Citation

If you use this wrapper in your work, please cite it (see CITATION.cff; GitHub renders a "Cite this repository" button):

Morão, H. (2026). worldbank-commodities (v0.8.1) [Software]. https://pypi.org/project/worldbank-commodities/

Please also credit the underlying data source, the World Bank Commodity Price Data (The Pink Sheet).

Data source & license

Data © The World Bank, published under CC BY 4.0. Source: https://www.worldbank.org/en/research/commodity-markets.

This wrapper code is released under the MIT License.