Beacon Data Lake Python API
Python wrapper for interacting with the Beacon Data Lake API. It discovers tables and datasets, inspects schemas, and ships a composable query builder that streams results straight into Pandas, GeoPandas, xarray, and on-disk formats such as (Geo)Parquet, NetCDF, and Zarr.
The full documentation lives at https://maris-development.github.io/beacon-py/.
Installation
pip install beacon-api
Beacon API supports Python 3.10+.
Quick start — connect
The examples below run against the public World Ocean Database (WOD) node at https://beacon-wod.maris.nl, so you can paste them straight into a notebook. Always pass a user_agent that identifies your application (and ideally a contact) so requests can be attributed on shared/public nodes.
from beacon_api import Client
client = Client(
"https://beacon-wod.maris.nl",
user_agent="my-app/1.0 (you@example.com)",
# jwt_token="<bearer token>", # for protected nodes
# basic_auth=("user", "pass"), # or HTTP basic auth
)
client.check_status() # probes /api/health and prints the Beacon version
Getting started: SQL
Already have SQL? Run it directly and materialize the result as a DataFrame:
df = client.sql_query(
"""
SELECT lon, lat, z, time, Temperature, Salinity
FROM "default"
WHERE time BETWEEN '2020-01-01T00:00:00' AND '2020-02-01T00:00:00'
"""
).to_pandas_dataframe()
print(df.head())
Getting started: JSON query builder
Prefer a fluent, typed builder? Start from a table and chain selects and filters. The builder and SQL paths share the same output helpers (to_pandas_dataframe(), to_parquet(), …).
tables = client.list_tables()
wod = tables["default"]
df = (
wod
.query()
.add_select_column("lon", alias="longitude")
.add_select_column("lat", alias="latitude")
.add_select_column("z", alias="depth")
.add_select_column("time")
.add_select_column("Temperature")
.add_select_column("Salinity")
.add_range_filter("time", "2020-01-01T00:00:00", "2020-02-01T00:00:00")
.to_pandas_dataframe()
)
print(df.head())
Going further
Explore tables & schemas
list_tables() returns DataTable helpers that already know their description, type, and schema:
tables = client.list_tables()
wod = tables["default"]
print(wod.get_table_description())
schema = wod.get_table_schema_arrow() # pyarrow.Schema
for field in schema:
print(field.name, field.type)
# get_table_schema() instead returns a plain dict[str, type]
print(wod.get_table_schema())
See Working with tables.
Datasets — query files directly
On Beacon ≥ 1.4.0, list_datasets() surfaces file-backed resources you can query without going through a logical table:
datasets = client.list_datasets(pattern="**/*.parquet", limit=10)
first = next(iter(datasets.values()))
print(first.get_file_name(), first.get_file_format())
df = first.query().add_select_column("lon").add_select_column("lat").to_pandas_dataframe()
More complex queries
The JSON builder supports range/equality/null/geospatial filters, boolean combinations, distinct, sorting, and a range of output formats:
df = (
wod
.query()
.add_select_column("lon")
.add_select_column("lat")
.add_select_column("time")
.add_select_column("Temperature")
.add_range_filter("time", "2020-01-01T00:00:00", "2020-06-30T23:59:59")
.add_range_filter("z", 0, 50)
.add_is_not_null_filter("Temperature")
.add_bbox_filter("lon", "lat", bbox=(-20, 40, -10, 55))
.add_sort("time", ascending=True)
.to_pandas_dataframe()
)
See Querying the Beacon Data Lake for the full builder reference and export helpers (to_geo_pandas_dataframe, to_parquet, to_netcdf, to_zarr, …).
Streaming large results
For result sets too large to buffer, sql_query_streaming() returns a PyArrow RecordBatchStreamReader (requires Beacon ≥ 1.5.0) that you can consume batch by batch:
reader = client.sql_query_streaming('SELECT lon, lat, z, time, Temperature FROM "default"')
for batch in reader:
# batch is a pyarrow.RecordBatch
print(batch.num_rows)
Issues
If you encounter any issues or have feature requests, please report them on the GitHub Issues page.
Development
This project is under active development. Contributions are welcome!
To generate the typings for the API, run:
stubgen beacon_api -o .
To build the wheel package, run:
python -m build
1 maintainer
Metadata
1 maintainer
Release files for beacon-api 1.3.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| beacon_api-1.3.4.tar.gz | 27.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| beacon_api-1.3.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 62.1 kB
Release files / beacon_api-1.3.4.tar.gz
| Download URL | beacon_api-1.3.4.tar.gz |
|---|---|
| Size | 27.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dd24429d9c597c234271b7ee301218823005fd27ce535c67faf8e6f7d0923cc5
|
|
BLAKE2b-256 checksum How to use checksums |
c93a5cbd9b94af6bd7241bc0ee4bccc0b60393100202a578b789b2fde0f50633
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / beacon_api-1.3.4-py3-none-any.whl
| Download URL | beacon_api-1.3.4-py3-none-any.whl |
|---|---|
| Size | 34.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0c191a9541b533cf52c66977c8ad172a6e2398c1220ad8f9e00229b2c469553b
|
|
BLAKE2b-256 checksum How to use checksums |
83894de2ae55f3665e73b12ccec74b816c04c59eff02d168eaae46e8241748fa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
