Python tools for the Aerocom project
Project description
About
pyaerocom is written and tested for Python >= 3.6 and for unix based systems. It provides tools for processing and plotting of data related to the AeroCom project.
This includes support for reading and processing of modeldata (e.g. AeroCom, EMEP), satellite data (e.g. MODIS, AATSR) and ground based observation datasets (e.g. AERONET, EBAS, EARLINET). In addition, pyaerocom provides tools for colocation and cross evaluation of different datasets using commonly used statistical metrics such as several biases, gross-errors, or correlation coefficients.
Main features
Reading routines for many ground based observation databases, such as:
AERONET Sun, SDA and Inversion products.
GHOST dataset (see e.g., Petetin et al., 2020 for more information).
Reading routines for level 3 gridded satellite observations, such as:
CALIPSO CALIOP Lidar observations.
Data harmonization tools following the CF conventions.
Intuitive data objects for analysis of gridded data and ungridded (point-cloud) observations.
Sophisticated colocation routines for model evaluation and intercomparison of observations.
Interfaces for conversion of data to data types of related data analysis libraries such as pandas, numpy, xarray or iris.
Data visualization tools and interfaces to common plotting libraries such as matplotlib or cartopy.
Tools for statistical analysis.
Toolbox for analysis of trends in time-series.
Tools to compute ensemble averages from multiple model outputs.
High-level tools for automated analyses of multi-model and multi-obs inter-comparison studies.
Usage examples
Processing of data for the new AeroCom Model Evaluation interface.
Processing and harmonization of observations for Aerosol Trends interface.
pyaerocom was used for the model evaluation study by Gliß et al., 2020.
pyaerocom was used for the trends analysis by Mortier et al., 2020.
AeroCom
The AeroCom-project (http://aerocom.met.no/) is an open international initiative of scientists interested in the advancement of the understanding of the global aerosol and its impact on climate. A large number of observations (including MODIS, POLDER, MISR, AVHHR, SEAWIFS, TOMS, AATSR, AERONET and surface concentrations) and results from more than 14 global models have been assembled to document and compare state of the art modeling of the global aerosol. A common protocol has been established and models are asked to make use of the AeroCom emission inventories for the year 2000 and preindustrial times. Results are documented via interactive websites which give access to 2D fields and standard comparisons to observations. Regular workshops are held to discuss findings and future directions.
This repository contains the AeroCom python tools which are / will be used to produce the standard AeroCom analyses shown at the AeroCom phase 2 interface (http://aerocom.met.no/cgi-bin/AeroCom/aerocom/surfobs_annualrs.pl)
At this point the tools are co-operational together with the IDL based aerocom-tools that cannot be made public because they use 3rd party libraries with a non GPL compatible license.
Citation
If you use pyaerocom for your research and find it useful, please consider citing pyaerocom. DOI’s and citation entries are available via Zenodo for recent releases here:
Website and code documentation
The official website including code documentation is hosted here:
Installation
You have several options to install pyaerocom, the first one is the easiest, but may not refer to the most recent (non-released) version of pyaerocom. So please check first, which version you are interested in.
Via conda
NOTE: This will install the latest release of pyaerocom.
It hence, may not include the most recent developments.
Requirements are installed automatically.
If you use conda as a package manager, the easiest way to install pyaerocom (and all requirements, see previous section) is to use the build provided in the nordicesmhub conda channel:
conda install -c conda-forge pyaerocom
This will install the latest release of pyaerocom including all requirements. Alternatively, you may install from source as described in the following.
NOTE: installation support via conda as described above is quite recent, so please let us know if you run into problems with the installation (best way to do this is by raising an issue here).
Via PyPi
NOTE: this will install the latest released version of pyaerocom, which is the same as distributed via conda-forge (see prev. point). However, installation via PyPi does not take care of any requirements (see below) but only installs pyaerocom:
pip install pyaerocom
Installing from source
If you use the conda packages manager, please make sure to activate the environment you want to install pyaerocom into. For more information about conda environments, see here.
Please make sure to install all requirements (see below) before installing pyaerocom from source. You can do that with the provided file pyaerocom_env.yml.
To install pyaerocom from source, please download and extract the latest release (or clone the repo) and install from the top-level directory (that contains a file setup.py) using:
python setup.py install
Alternatively, if you plan to apply local changes to the pyaerocom source code, you may install in development mode:
python setup.py develop
You may also download and extract (or clone) the GitHub repo to install the very latest (not yet released) version of pyaerocom.
Requirements
A list of all requirements is provided in file pyaerocom_env.yml.
Installing all requirements
NOTE: You can skip this section if you install the latest conda-forge release (more infos below under Installation).
We recommend using the Anaconda Python 3.7 distribution (or Miniconda, if you want to save disk space) and to use the conda package manager to install the requirements.
If you use Anaconda as a package manager, you can install all requirements (specified in previous section) into a new environment using the provided pyaerocom_env.yml file:
conda env create -n pya -f pyaerocom_env.yml
This will create a new conda environment called pya which can be activated using:
conda activate pya
Alternatively, you can include the requirements into an existing environment. First, activate the existing environment, and then install the dependencies using:
conda env update -f=pyaerocom_env.yml
Access to users database
Please follow the instructions provided here, to retrieve access to the AeroCom users database:
https://wiki.met.no/aerocom/data_retrieval
NOTE: the users database does not contain any ground based observational data (such as EBAS, AERONET, etc.) but only the AeroCom model data available in the database as well as some gridded level 3 satellite datasets which may be used for model evaluation. Once you have access to the user database you may mount the file-system locally (e.g. via sshfs and register the data-paths you need in pyaerocom, for details see tutorials, more info below).
Getting started
After installing pyaerocom, open your python executable and try to import pyaerocom:
import pyaerocom as pya
To get started, please checkout the tutorials.
Tutorials (Jupyter notebooks)
A number of tutorial jupyter notebooks can be found in the pyaerocom-tutorials repo.
Remark for Windows users
pyaerocom is not tested on Windows systems and may only work in parts and thus some features may not work on Windows machines at the moment. In particular, features that rely on and are built upon access to the AeroCom database servers and automatic database path navigation. This includes the automised reading of gridded and ungridded data using either of the pre-defined path infrastuctures (e.g. check out paths.ini or paths_user_server.ini). However, you may still define file locations in your Python scripts yourself and use the more low-level features for reading the data. Windows support will be provided soon. Please let us know if you intend to use pyaerocom on a Windows machine so that we can consider adjusting our priorities, or also if you have any questions related to the usage.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pyaerocom-0.10.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5dcd1bacb04b730bb35e43cd8f89dffa2fa7ea109d5b9b13302d2e59ab0df3e5 |
|
MD5 | 5c11495417d94265faae94e372f1f582 |
|
BLAKE2b-256 | f8aa9added4bdcc9bbeca88fc5ed3fe496cbd27bc77d630d1bf4538d425c6785 |