ORLEN wholesale fuel-price scraper

This repository downloads the public price data shown on ORLEN's Hurtowe ceny paliw page. It covers every section currently visible there:

• motor fuels (all products returned by the API),
• wholesale autogas/LPG (all 16 regions),
• heavy fuel oil,
• aviation fuels (all locations).

It calls the page's JSON API sequentially. The default random 2–4 second gap, 24-hour disk cache, incremental CSV/state tracking, Retry-After handling, exponential backoff, and atomic output make long runs polite and resumable. Do not run many copies in parallel.

Installation

From GitHub:

python -m pip install git+https://github.com/solek13/orlen.git

From a local checkout:

python -m pip install -e .

After installation, the command-line tool is available as orlen-prices.

Usage

Python 3.10+ is sufficient; there are no third-party dependencies.

# After installation
orlen-prices --year 2026 --output data\prices.csv

# Current year (the default)
python .\scrape_orlen.py

# A particular year
python .\scrape_orlen.py --year 2025

# Everything exposed by the API (slowest, especially LPG)
python .\scrape_orlen.py --all

# January 1, 2013 through today (also accepts: --since 2013)
python .\scrape_orlen.py --from-year 2013

# Choose the destination
python .\scrape_orlen.py --year 2026 --output data\prices.csv

The default files are data/orlen_prices_<year>.csv, data/orlen_prices_since_<year>.csv, and data/orlen_prices_all.csv. Each CSV has a small .csv.state.json sidecar. It records the date range already checked for every product, including products which returned no rows. A later run loads and merges the existing CSV, requests only dates after the recorded coverage, and skips LPG effective dates already present. The state is bound to a hash of the CSV so deleting or modifying the data cannot silently cause ranges to be skipped.

Product lists and LPG date indexes are checked through the 24-hour cache so new products/dates can still be discovered without repeatedly downloading price history. Use --refresh only when those missing-range requests really need to bypass the cache. An interrupted run can simply be started again; completed requests remain in the response cache.

The all-history run deliberately stays single-threaded. Regular, heavy-oil, and aviation history need only a few range requests. ORLEN exposes LPG history one effective date at a time, so that part can take several minutes. Progress is printed to stderr.

The CSV contains category/product metadata, effective and publication dates, price, frontend display unit, location (where applicable), exact source URL, and fetch timestamp. ORLEN's API currently sends a null unit; unit is filled from the units displayed in the corresponding ORLEN frontend tables.

Library functions

from scrape_orlen import scrape_all, scrape_since, scrape_year, update_csv, write_csv

rows = scrape_year()          # current year
# rows = scrape_all()         # complete available history
# rows = scrape_since(2013)   # 2013 through today
write_csv(rows, "prices.csv")

# Preferred for scheduled runs: merges existing rows and downloads only gaps.
rows = update_csv("prices.csv")

Tests

python -m unittest -v

The tests use a fake API and do not contact ORLEN.

Packaging

Build the source distribution and wheel locally:

python -m pip install -e .[dev]
python -m build
python -m twine check dist/*

Installing by package name, for example python -m pip install orlen-price-scraper, requires uploading a release to PyPI first.