Wetterdienst - Open weather data for humans
“What do we want? Climate Justice! When do we want it? Now!” - FFF
WARNING
This library is a work in progress!
Breaking changes should be expected until a 1.0 release, so version pinning is recommended.
CI
Meta
Downloads
Citation
Introduction
Overview
Welcome to Wetterdienst, your friendly weather service library for Python.
We are a group of like-minded people trying to make access to weather data in Python feel like a warm summer breeze, similar to other projects like rdwd for the R language, which originally drew our interest in this project. Our long-term goal is to provide access to multiple weather services as well as other related agencies such as river measurements. With wetterdienst we try to use modern Python technologies all over the place. The library is based on polars (we <3 pandas, it is still part of some IO processes) across the board, uses Poetry for package administration and GitHub Actions for all things CI. Our users are an important part of the development as we are not currently using the data we are providing and only implement what we think would be the best. Therefore contributions and feedback whether it be data related or library related are very welcome! Just hand in a PR or Issue if you think we should include a new feature or data source.
Data
For an overview of the data we have currently made available and under which license it is published take a look at the data section. Detailed information on datasets and parameters is given at the coverage subsection. Licenses and usage requirements may differ for each provider so check this out before including the data in your project to be sure that you fulfill copyright requirements!
Features
APIs for stations and values
Get stations nearby a selected location
Define your request by arguments such as parameter, period, resolution, start date, end date
Define general settings in Settings context
Command line interface
Web-API via FastAPI
Rich UI features like wetterdienst explorer
Run SQL queries on the results
Export results to databases and other data sinks
Public Docker image
Interpolation and Summary of station values
Setup
Native
Via PyPi (standard):
pip install wetterdienstVia Github (most recent):
pip install git+https://github.com/earthobservations/wetterdienstThere are some extras available for wetterdienst. Use them like:
pip install wetterdienst[sql]docs: Install the Sphinx documentation generator.
ipython: Install iPython stack.
export: Install openpyxl for Excel export and pyarrow for writing files in Feather- and Parquet-format.
sql: Install DuckDB for querying data using SQL.
duckdb: Install support for DuckDB.
influxdb: Install support for InfluxDB.
cratedb: Install support for CrateDB.
mysql: Install support for MySQL.
postgresql: Install support for PostgreSQL.
interpolation: Install support for station interpolation.
In order to check the installation, invoke:
wetterdienst --helpDocker
Docker images for each stable release will get pushed to GitHub Container Registry.
wetterdienst serves a full environment, including all of the optional dependencies of Wetterdienst.
Pull the Docker image:
docker pull ghcr.io/earthobservations/wetterdienstLibrary
Use the latest stable version of wetterdienst:
$ docker run -ti ghcr.io/earthobservations/wetterdienst
Python 3.8.5 (default, Sep 10 2020, 16:58:22)
[GCC 8.3.0] on linuximport wetterdienst
wetterdienst.__version__Command line script
The wetterdienst command is also available:
# Make an alias to use it conveniently from your shell.
alias wetterdienst='docker run -ti ghcr.io/earthobservations/wetterdienst wetterdienst'
wetterdienst --help
wetterdienst --version
wetterdienst infoRaspberry Pi / LINUX ARM
Running wetterdienst on Raspberry Pi, you need to install numpy and lxml prior to installing wetterdienst by running the following lines:
# not all installations may be required to get lxml running
sudo apt-get install gfortran
sudo apt-get install libopenblas-base
sudo apt-get install libopenblas-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install python3-lxmlAdditionally expanding the Swap to 2048 mb may be required and can be done via swap-file:
sudo nano /etc/dphys-swapfileThanks chr-sto for reporting back to us!
Example
Task: Get historical climate summary for two German stations between 1990 and 2020
Library
>>> import polars as pl
>>> _ = pl.Config.set_tbl_hide_dataframe_shape(True)
>>> from wetterdienst import Settings
>>> from wetterdienst.provider.dwd.observation import DwdObservationRequest
>>> settings = Settings( # default
... ts_shape="long", # tidy data
... ts_humanize=True, # humanized parameters
... ts_si_units=True # convert values to SI units
... )
>>> request = DwdObservationRequest(
... parameter="climate_summary",
... resolution="daily",
... start_date="1990-01-01", # if not given timezone defaulted to UTC
... end_date="2020-01-01", # if not given timezone defaulted to UTC
... settings=settings
... ).filter_by_station_id(station_id=(1048, 4411))
>>> stations = request.df
>>> stations.head()
┌────────────┬──────────────┬──────────────┬──────────┬───────────┬────────┬─────────────┬─────────┐
│ station_id ┆ start_date ┆ end_date ┆ latitude ┆ longitude ┆ height ┆ name ┆ state │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ datetime[μs, ┆ datetime[μs, ┆ f64 ┆ f64 ┆ f64 ┆ str ┆ str │
│ ┆ UTC] ┆ UTC] ┆ ┆ ┆ ┆ ┆ │
╞════════════╪══════════════╪══════════════╪══════════╪═══════════╪════════╪═════════════╪═════════╡
│ 01048 ┆ 1934-01-01 ┆ ... ┆ 51.1278 ┆ 13.7543 ┆ 228.0 ┆ Dresden-Klo ┆ Sachsen │
│ ┆ 00:00:00 UTC ┆ 00:00:00 UTC ┆ ┆ ┆ ┆ tzsche ┆ │
│ 04411 ┆ 1979-12-01 ┆ ... ┆ 49.9195 ┆ 8.9672 ┆ 155.0 ┆ Schaafheim- ┆ Hessen │
│ ┆ 00:00:00 UTC ┆ 00:00:00 UTC ┆ ┆ ┆ ┆ Schlierbach ┆ │
└────────────┴──────────────┴──────────────┴──────────┴───────────┴────────┴─────────────┴─────────┘
>>> values = request.values.all().df
>>> values.head()
┌────────────┬─────────────────┬───────────────────┬─────────────────────────┬───────┬─────────┐
│ station_id ┆ dataset ┆ parameter ┆ date ┆ value ┆ quality │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ str ┆ str ┆ datetime[μs, UTC] ┆ f64 ┆ f64 │
╞════════════╪═════════════════╪═══════════════════╪═════════════════════════╪═══════╪═════════╡
│ 01048 ┆ climate_summary ┆ cloud_cover_total ┆ 1990-01-01 00:00:00 UTC ┆ 100.0 ┆ 10.0 │
│ 01048 ┆ climate_summary ┆ cloud_cover_total ┆ 1990-01-02 00:00:00 UTC ┆ 100.0 ┆ 10.0 │
│ 01048 ┆ climate_summary ┆ cloud_cover_total ┆ 1990-01-03 00:00:00 UTC ┆ 91.25 ┆ 10.0 │
│ 01048 ┆ climate_summary ┆ cloud_cover_total ┆ 1990-01-04 00:00:00 UTC ┆ 28.75 ┆ 10.0 │
│ 01048 ┆ climate_summary ┆ cloud_cover_total ┆ 1990-01-05 00:00:00 UTC ┆ 91.25 ┆ 10.0 │
└────────────┴─────────────────┴───────────────────┴─────────────────────────┴───────┴─────────┘values.to_pandas() # to get a pandas DataFrame and e.g. create some matplotlib plotsClient
# Get list of all stations for daily climate summary data in JSON format
wetterdienst stations --provider=dwd --network=observation --parameter=kl --resolution=daily --all
# Get daily climate summary data for specific stations
wetterdienst values --provider=dwd --network=observation --station=1048,4411 --parameter=kl --resolution=dailyFurther examples (code samples) can be found in the examples folder.
Acknowledgements
We want to acknowledge all environmental agencies which provide their data open and free of charge first and foremost for the sake of endless research possibilities.
We want to acknowledge Jetbrains and the Jetbrains OSS Team for providing us with licenses for Pycharm Pro, which we are using for the development.
We want to acknowledge all contributors for being part of the improvements to this library that make it better and better every day.
Important Links
Full documentation: https://wetterdienst.readthedocs.io/
Contribution: https://wetterdienst.readthedocs.io/en/latest/contribution/
Known Issues: https://wetterdienst.readthedocs.io/en/latest/known_issues/
Changelog: https://wetterdienst.readthedocs.io/en/latest/changelog.html
Examples (runnable scripts): https://github.com/earthobservations/wetterdienst/tree/main/examples
Benchmarks: https://github.com/earthobservations/wetterdienst/tree/main/benchmarks
1 maintainer
Metadata
1 maintainer
Release files for wetterdienst 0.86.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| wetterdienst-0.86.0.tar.gz | 208.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| wetterdienst-0.86.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 463.5 kB
Release files / wetterdienst-0.86.0.tar.gz
| Download URL | wetterdienst-0.86.0.tar.gz |
|---|---|
| Size | 208.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6029c282f889faf88603f250100e73db77acb27f736b558ad9c0a8792a4187b6
|
|
BLAKE2b-256 checksum How to use checksums |
8c7f26d22cb982d6f82ff255afc930940ea971a8698a4bff39610f6f60a4385e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.3 CPython/3.10.12 Linux/6.5.0-1021-azure
|
Release files / wetterdienst-0.86.0-py3-none-any.whl
| Download URL | wetterdienst-0.86.0-py3-none-any.whl |
|---|---|
| Size | 255.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
dc0cb8ca3090c3db458bf527b5108f1e69aa72916e01c837d48e2f781b872ff6
|
|
BLAKE2b-256 checksum How to use checksums |
6624b48d46425c7904ef3fbde3eeed0ce90d39dcbc47d44ca1ab85155895ea14
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.3 CPython/3.10.12 Linux/6.5.0-1021-azure
|
