osmsg

OpenStreetMap Stats Generator. A tiny CLI (and Python library) that turns OSM history into per-user counts of nodes, ways, and relations created, modified, or deleted, written to parquet, csv, json, markdown, or Postgres.

A Project of OSGeo Nepal.

What does it do?

Per-user create/modify/delete counts over any time window.
Tag and hashtag breakdowns (e.g. building, #hotosm).
Country and custom-boundary filters via Geofabrik.
Cron-friendly resume with --update.
One-command setup: osmsg --insert loads all history into your store, osmsg --update keeps it current.
Outputs you can query: parquet, csv, json, markdown, DuckDB, Postgres.
Cloud-native history: months covered by a published parquet dataset are read remotely.

Install

Pick the one that fits how you work.

uvx --from osmsg osmsg --last hour       # zero-install, one-shot run
pip install osmsg                        # into your project
uv tool install osmsg                    # standalone CLI
docker run --rm -v "$PWD:/work" -w /work ghcr.io/osgeonepal/osmsg:latest --last hour

uvx can run osmsg in a throwaway environment , no install, no virtualenv to manage. Works with any flag combination, e.g. uvx --from osmsg osmsg --last hour --tags building --summary -f parquet -f markdown.

Planned install channels (not yet published): conda-forge (conda install -c conda-forge osmsg) and a Homebrew tap (brew install osgeonepal/tap/osmsg).

On Windows, download osmsg.exe from the latest release and double-click it to open the desktop app. Pick a Quick range (last hour, day, week, month, year, or all time) or type your own dates, set the options, click Compute, and open the output folder. The CLI below is for macOS, Linux, and pip/uv users.

Quick start

osmsg --last hour                        # planet, last hour
osmsg --last day --tags building         # last day with a tag breakdown
osmsg --hashtags hotosm --last day       # only changesets tagged #hotosm

That's it. A stats.duckdb and a stats.parquet show up in your current folder.

Set up a full history store

Two commands give you a complete, self-updating store. The first loads all of OSM history from the published dataset and records where to resume; the second catches up to now and runs on a schedule.

osmsg --insert            # load all history into stats.duckdb, then exit
osmsg --update            # catch up to now (repeat on cron)

osmsg clears the multi-week backlog on day diffs, then refines to finer diffs as the store stays current. For near-real-time, run osmsg --update --url minute.

Pick your store with one flag. DuckDB is the default (stats.duckdb); add a DSN for Postgres:

osmsg --insert --psql-dsn "postgresql://user:pass@localhost/osmsg"
osmsg --update --psql-dsn "postgresql://user:pass@localhost/osmsg"

Load only a slice with --start/--end; --update then continues from the end of that slice:

osmsg --insert --start 2020-01-01 --end 2023-01-01

Already have the planet files? Insert from them directly:

osmsg --insert --osh-file history-latest.osh.pbf --changeset-file changesets-latest.osm.bz2

Tutorials

1. Stats for a country

osmsg --country nepal --last day

--country resolves through Geofabrik and needs an OSM account. Set OSM_USERNAME and OSM_PASSWORD in your shell or a .env file:

export OSM_USERNAME=you
export OSM_PASSWORD=secret

2. A custom date range with summaries

osmsg --start "2026-04-01" --end "2026-04-08" \
      --tags building --tags highway --summary

--summary adds a daily rollup file alongside the per-changeset stats.

3. Run on a schedule

osmsg --country nepal --update           # picks up where the last run stopped

Drop that into cron or a GitHub Actions schedule. State is stored inside the DuckDB file, so reruns are safe.

4. Query the output

duckdb stats.duckdb -c "SELECT username, SUM(nodes_created) AS n
                        FROM users JOIN changeset_stats USING (uid)
                        GROUP BY username ORDER BY n DESC LIMIT 10"

Same schema in DuckDB and Postgres: users, changesets, changeset_stats, state.

5. Run the API

Push stats into Postgres, then start the Litestar API. The API dependencies live in the api dependency group and the api/ package runs from a repo checkout (it is not part of the published wheel), so run it from a clone:

git clone https://github.com/osgeonepal/osmsg && cd osmsg
uv sync --group api
uv run osmsg --last day --format psql --psql-dsn "postgresql://user:pass@localhost/osmsg"
uv run --group api litestar --app api.app:app run --host 0.0.0.0 --port 8000

GET /health
GET /api/v2/hashtag/hotosm/summary
GET /api/v2/hashtag/hotosm/leaderboard
GET /docs

For self-hosting with Docker Compose and systemd, see docs/infra.md.

6. Use it as a library

from datetime import datetime, UTC
from osmsg import RunConfig, run

result = run(RunConfig(
    name="nepal",
    countries=["nepal"],
    start_date=datetime(2026, 4, 25, tzinfo=UTC),
    end_date=datetime(2026, 4, 26, tzinfo=UTC),
))
print(result["files"]["parquet"])

Same pipeline as the CLI.

7. Long flag lists? Use a config

osmsg --config nepal.yaml

Each option is a YAML key written with its underscore name: output_dir, history_url, all_stats, formats, psql_dsn, and so on (not the dashed flag). See docs/Manual.md.

Output formats

Every run writes stats.duckdb (or <--name>.duckdb) plus the formats you ask for via -f parquet|csv|json|markdown|psql. Parquet is the default. Open it with duckdb, polars, pandas, anything.

Rerunning the same query with a different -f re-exports from the existing <name>.duckdb instead of refetching, so adding a format is instant. Pass --overwrite to force a fresh recompute.

Configuration

Every meaningful flag has a matching OSMSG_* env var so the CLI, a .env file, and a docker-compose environment: block all reach the same setting. CLI flag wins over env var.

CLI flag	Env var	Default	Notes
`--name`	`OSMSG_NAME`	`stats`	Output basename; sets `<name>.duckdb`.
`--country`	`OSMSG_COUNTRY`	unset	Geofabrik region id(s). Comma-separated when set via env.
`--boundary`	`OSMSG_BOUNDARY`	unset	GeoJSON path or inline GeoJSON.
`--url`	`OSMSG_URL`	`minute`	`minute`/`hour`/`day` shortcut or full URL. Comma-separated when set via env.
`--workers`	`OSMSG_WORKERS`	cpu count	Parallel parse workers.
`--cache-dir`	`OSMSG_CACHE_DIR`	platform cache	Where downloaded OSM files are kept across runs.
`--output-dir`	`OSMSG_OUTPUT_DIR`	`.`	Where `<name>.duckdb` and exports are written.
`--format` / `-f`	`OSMSG_FORMAT`	`parquet`	Repeat for multiple. Comma-separated when set via env.
`--overwrite`	(none)	off	Recompute even if `<name>.duckdb` already holds this exact query.
`--psql-dsn`	`OSMSG_PSQL_DSN`	unset	libpq DSN for `-f psql`.
`--psql-bulk`	`OSMSG_PSQL_BULK`	off	Faster first full load to Postgres.
`--history` / `--no-history`	`OSMSG_HISTORY`	on	Read covered months from the published dataset.
`--history-url`	`OSMSG_HISTORY_URL`	`osmsg-history`	Published dataset location.
`--insert`	(none)	off	Load history into the store and seed resume, then exit. No window loads all of it.
`--seed-only`	(none)	off	With `--insert`: seed resume state at the published frontier without loading rows.
`--osh-file` / `--changeset-file`	(none)	unset	Insert from local planet history + changeset files.
`--max-update-window-hours`	`OSMSG_MAX_UPDATE_WINDOW_HOURS`	unset	Cap each `--update` run to this many hours so a large backlog catches up over successive runs.
`--changeset-pad-hours`	`OSMSG_CHANGESET_PAD_HOURS`	`1`	See below.
(auto-bootstrap on `--update`)	`OSMSG_BOOTSTRAP`	`hour`	`hour`, `day`, or `week`. Used when `--update` runs against an empty DB.
(auto-bootstrap on `--update`)	`OSMSG_BOOTSTRAP_DAYS`	unset	Integer N; overrides `OSMSG_BOOTSTRAP`.
OSM credentials (Geofabrik)	`OSM_USERNAME`, `OSM_PASSWORD`	unset	Required only when a Geofabrik URL is in use.

A .env file at the working directory is loaded automatically.

Maintainers

Generating and publishing the history dataset is the osmsg maintain group:

osmsg maintain month 2026-06 --repo osgeonepal/osmsg-history   # append one finished month
osmsg maintain month 2026-06 --no-upload                       # generate locally, review, upload later
osmsg maintain convert history.osh.pbf changesets.osm.bz2 2005-01-01 2026-06-01 work --parts 24
osmsg maintain publish work/out --repo osgeonepal/osmsg-history
osmsg maintain refresh --artifact-dir /srv/osmsg/artifact    # pull the newest published month locally
osmsg maintain prune-pg --psql-dsn "postgresql://user:pass@localhost/osmsg"   # drop rows now covered by history
osmsg maintain check --psql-dsn "postgresql://user:pass@localhost/osmsg" --fix # backfill stub changesets

Documentation

Contributing

Pull requests are welcome. Quick path:

git clone https://github.com/osgeonepal/osmsg && cd osmsg
git switch develop
uv sync
uv run pre-commit install
uv run pytest -m "not network"

Please read CONTRIBUTING.md and the Code of Conduct before opening a PR. Use Conventional Commits (cz commit).

License

MIT © OSGeo Nepal contributors.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

osmsg-1.3.3.tar.gz (87.9 kB view details)

Uploaded Sep 6, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

osmsg-1.3.3-py3-none-any.whl (105.4 kB view details)

Uploaded Sep 6, 2026 Python 3

File details

Details for the file osmsg-1.3.3.tar.gz.

File metadata

Download URL: osmsg-1.3.3.tar.gz
Upload date: Sep 6, 2026
Size: 87.9 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.12.10 {"installer":{"name":"uv","version":"0.12.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for osmsg-1.3.3.tar.gz
Algorithm	Hash digest
SHA256	`5f9a92fbfeb8e2e549fe9da26299c03b4f83e2cf75f16a4a41fddbe4eb2f5f04`
MD5	`3482d2a6c2991422fb335d13b5c20f67`
BLAKE2b-256	`4f857fed6b63d9e609922182b6c199ffdb04bf43518be4d91218e23b3ae44cab`

See more details on using hashes here.

File details

Details for the file osmsg-1.3.3-py3-none-any.whl.

File metadata

Download URL: osmsg-1.3.3-py3-none-any.whl
Upload date: Sep 6, 2026
Size: 105.4 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.12.10 {"installer":{"name":"uv","version":"0.12.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for osmsg-1.3.3-py3-none-any.whl
Algorithm	Hash digest
SHA256	`471c872bf324e7648e3bc881638762260b35a6e3b4d49a80384e4f8fa0a18594`
MD5	`5e8a3e85e33d184fd9b31f511af34bc5`
BLAKE2b-256	`6348bc965deb3d50ffe2f0d3499f20964813dffcabac0d525484dcbcc09057b7`