This release is a pre-release and may not be stable for production use.
Recommendations for Safe Lossy Compression of weather and climate data
What lossy compression is safe when using lossy compression on weather and climate data?
This repository contains community-provided recommendations that are automatically compiled into one machine-readable recommendations.yaml file.
These recommendations can then be used by external tools to automatically recommend appropriate safety requirements for compressing various weather and climate data.
This repository also provides the compression-recommendations Python package for loading these recommendations and inspecting them in the strongly typed Recommendations data structure.
Furthermore, we provide the following integrations:
compression-requirement-checkschecks whether safety requirements are upheld by a lossy-decompressed reconstruction of the original data.compression-requirement-safeguardstranslates safety requirements into compression safeguards that can be wrapped around any compressor to guarantee that the requirements are fulfilled.
Contributing: What should be recommended?
That is up to you, the community, to decide!
- You can add comments to any recommendation to elaborate or discuss different requirements.
- You can propose edits to any recommendation, which we accept once a new consensus has been reached.
- You can create new recommendations for new variables or to specialise existing requirements for specific cases.
- You can merge multiple recommendations to generalise them.
- You can propose new filters to select which recommendations should apply, or new kinds of requirements.
Glossary
Compression : Reducing the number of bits needed to store some data.
Lossy Compression : Compression that may only produce an approximation of the original data during decompression.
Filter : Criteria, or a composition of multiple criteria, that select which requirements should apply.
Safety Requirement : A property, e.g. an error bound, or a composition of multiple properties, that the lossy-compressed data must fulfil with respect to the original uncompressed data.
Citation
Please refer to the CITATION.cff file and refer to https://citation-file-format.github.io to extract the citation in a format of your choice.
License
Licensed under the Mozilla Public License, Version 2.0 (LICENSE or https://www.mozilla.org/en-US/MPL/2.0/).
Funding
The outline for the recommendations have been developed as part of ESiWACE3, the third phase of the Centre of Excellence in Simulation of Weather and Climate in Europe.
Funded by the European Union. This work has received funding from the European High Performance Computing Joint Undertaking (JU) under grant agreement No 101093054.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file compression_requirement_safeguards-0.1.0a1.tar.gz.
File metadata
- Download URL: compression_requirement_safeguards-0.1.0a1.tar.gz
- Upload date:
- Size: 11.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2172fe9e04650945dce58090fd3fbe4bcf3ea9c93d74350b97cd99da5445ce7b
|
|
| MD5 |
695b295f6a6c256b83de1e76c6be3cc8
|
|
| BLAKE2b-256 |
d9bd3e96ca837a70c62c7df22059b8fc9d93065100d0678cbe4591d2c8786dd7
|
Provenance
The following attestation bundles were made for compression_requirement_safeguards-0.1.0a1.tar.gz:
Publisher:
ci.yml on juntyr/compression-recommendations
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
compression_requirement_safeguards-0.1.0a1.tar.gz -
Subject digest:
2172fe9e04650945dce58090fd3fbe4bcf3ea9c93d74350b97cd99da5445ce7b - Sigstore transparency entry: 2839181237
- Sigstore integration time:
-
Permalink:
juntyr/compression-recommendations@b6488392998ba194157af5360a1554d5c2b1f5f3 -
Branch / Tag:
refs/tags/0.1.0a1.post1 - Owner: https://github.com/juntyr
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
ci.yml@b6488392998ba194157af5360a1554d5c2b1f5f3 -
Trigger Event:
release
-
Statement type:
File details
Details for the file compression_requirement_safeguards-0.1.0a1-py3-none-any.whl.
File metadata
- Download URL: compression_requirement_safeguards-0.1.0a1-py3-none-any.whl
- Upload date:
- Size: 11.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d42e66431e0ef6c374126f13aca94b32e34245a5b3ba0178afe3d967b6fa54f2
|
|
| MD5 |
7fc9c840d4de1bb42ff1ec2771f88470
|
|
| BLAKE2b-256 |
88b78971fecb1be7fd320a02e72b19402938660530836f4b79c8d6a410b3e3e3
|
Provenance
The following attestation bundles were made for compression_requirement_safeguards-0.1.0a1-py3-none-any.whl:
Publisher:
ci.yml on juntyr/compression-recommendations
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
compression_requirement_safeguards-0.1.0a1-py3-none-any.whl -
Subject digest:
d42e66431e0ef6c374126f13aca94b32e34245a5b3ba0178afe3d967b6fa54f2 - Sigstore transparency entry: 2839181316
- Sigstore integration time:
-
Permalink:
juntyr/compression-recommendations@b6488392998ba194157af5360a1554d5c2b1f5f3 -
Branch / Tag:
refs/tags/0.1.0a1.post1 - Owner: https://github.com/juntyr
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
ci.yml@b6488392998ba194157af5360a1554d5c2b1f5f3 -
Trigger Event:
release
-
Statement type:
