publicdata-au

Query and download Australian government open data from publicdata.au. publicdata.au republishes datasets that governments already publish under open licences, keeps every version at a URL that never changes, and serves each one in eleven formats with a query API.

This package works for every dataset the site serves, named by its slug, so a dataset added to the site needs no new release.

pip install publicdata-au            # queries and downloads, no dependencies
pip install "publicdata-au[pandas]"  # adds read() into a pandas DataFrame
pip install "publicdata-au[duckdb]"  # adds connect(), a DuckDB connection to a version

Find a dataset

import publicdata_au as pd_au

pd_au.datasets("road crashes")  # slug, title, publisher, licence and page of each match
pd_au.versions("au-road-deaths")  # every version kept, newest first

Query rows and totals

from publicdata_au import gte, in_

deaths = pd_au.rows(
    "au-road-deaths",
    {"state": in_("QLD", "NSW"), "year": gte(2020)},
    select=["state", "year", "road_user"],
    order="year.desc",
    all=True,
)
deaths.version  # the version the rows came from
deaths.attribution  # the attribution the publisher's licence requires
deaths.to_pandas()

pd_au.aggregate("au-road-deaths", group="state", metric="count", where={"year": 2025})

A plain value must match exactly, a list matches any of its values and None matches a blank or suppressed cell. The filters are eq, neq, gt, gte, lt, lte, like, ilike, in_, is_null and not_.

Without version= an answer comes from the newest version and changes when the publisher releases again. Pass a date from versions() for an answer that never changes.

Whole tables

df = pd_au.read("au-road-deaths")  # needs the [pandas] extra
df.attrs["publicdata"]  # the version, licence and attribution the file itself carries
pd_au.download(
    "au-road-deaths", "csv"
)  # parquet, csv, csv.gz, json, ndjson, sqlite, duckdb, xlsx, arrow, geojson, gpkg

Query a version in place

Every version has a DuckDB file, and connect() attaches it read-only over HTTPS. Only the blocks a query touches are read, so a count over millions of rows runs without a download.

con = pd_au.connect("au-road-deaths")  # needs the [duckdb] extra
con.sql("SELECT state, count(*) FROM records GROUP BY 1").df()
con.publicdata  # the version, its URL and its licence

A database such as G-NAF is one DuckDB file holding every table, the keys between them and the publisher's views, with one Parquet file per table beside it.

pd_au.tables("gnaf")  # every table with its fields, keys and references
con = pd_au.connect("gnaf")
con.sql("SELECT postcode, count(*) AS n FROM address_view GROUP BY 1 ORDER BY 2 DESC LIMIT 10").df()
pd_au.read("gnaf", table="locality")  # one table as a pandas DataFrame
pd_au.download("gnaf", table="state")  # one table as Parquet

Files have no rate limit. The query API allows 60 requests in 10 seconds from one address, and this package waits and retries when it answers 429.

Licence and attribution

The data is under each publisher's own licence, which requires the attribution string that every answer carries. Please also name publicdata.au and link to the version you used. publicdata.au is an independent republication, and the publishers have not endorsed it.

The package itself is under the MIT licence.