Skip to main content

hydropandas module by Artesia

Project description

Artesia

hydropandas Codacy Badge Codacy Badge Documentation Status PyPi Binder

hydropandas

The Hydropandas package allows users to manipulate data using all of the wonderful features included in pandas extented with custom methods and attributes related to the timeseries. The hydropandas module extends pandas.DataFrame with extra functionality and stores metadata related to the type of measurements.

Installation

Install the module with pip:

pip install hydropandas

Hydropandas requires numpy, scipy, matplotlib, pandas, geopandas, requests and zeep.

For some functionality additional packages are required:

  • pastastore: for reading or storing data from PastaStore
  • bokeh, branca, folium: for interactive maps
  • flopy: for reading data from MODFLOW models
  • xarray: for loading data from REGIS

For installing in development mode, clone the repository and install by typing pip install -e . from the module root directory.

If you have trouble installing hydropandas, refer to the Dependencies section below.

Example usage

Importing a single DINO csv file:

import hydropandas as hpd
fname = './tests/data/2019-Dino-test/Grondwaterstanden_Put/B33F0080001_1.csv'
gw = hpd.GroundwaterObs.from_dino(fname=fname, verbose=True)

Or for a zipfile:

import hydropandas as hpd
dinozip = './tests/data/2019-Dino-test/dino.zip'
dino_gw = hpd.ObsCollection.from_dino(dirname=dinozip,
                                      subdir='Grondwaterstanden_Put',
                                      suffix='1.csv',
                                      ObsClass=hpd.GroundwaterObs,
                                      keep_all_obs=False,
                                      verbose=False)

The Obs class

The Obs class holds the measurements and metadata for one timeseries. There are currently 5 specific Obs classes for different types of measurements:

  • GroundwaterObs: for groundwater measurements
  • GroundwaterQualityObs: for groundwater quality measurements
  • WaterlvlObs: for surface water level measurements
  • ModelObs: for "observations" from a MODFLOW model
  • KnmiObs: for (daily) KNMI observations

Each of these Obs classes is essentially a pandas DataFrame with additional methods and attributes related to the type of measurement that it holds. The classes also contain specific methods to read data from specific sources.

The ObsCollection class

The ObsCollection class, as the name implies, represents a collection of Obs classes, e.g. 10 timeseries of the groundwater level in a certain area. The ObsCollection is also a pandas DataFrame in which each timeseries is stored in a different row. Each row contains metadata (e.g. latitude and longitude of the observation point) and the Obs object (DataFrame) that holds the measurements. It is recommended to let an ObsCollection contain only one Obs type, e.g. to create an ObsCollection for 10 GroundwaterObs, and a separate ObsCollection for 5 KnmiObs.

Like the Obs class, the ObsCollection class contains a bunch of methods for reading data from different sources. See the next section for supported data sources.

Supported data sources

Currently supported datasources that can be read:

  • FEWS PI-XML
  • DINO csv
  • WISKI csv
  • Pastas projects (deprecated)
  • Pastastore, for managing Pastas timeseries and models
  • PyStore, a fast datastore for pandas timeseries
  • Arctic, a timeseries / dataframe database that sits atop MongoDB
  • KNMI data
  • MODFLOW groundwater models
  • IMOD groundwater models

An ObsCollection can be exported to:

  • Shapefile
  • Pastas projects (deprecated)
  • Pastastore
  • Arctic
  • Pystore

Dependencies

Hydropandas (indirectly) uses some packages that cannot be installed automatically with pip on Windows. These packages are:

  • GDAL
  • Fiona
  • Shapely
  • Python-snappy
  • Fastparquet

If you do not have these packages already it is recommended to first try installing them with conda install <pkg>. Otherwise, read the instructions below how to install them manually.

Download the packages from Christoph Gohlke's website. Use CTRL+F to find the download link on the page. Be sure to download the correct version of the package. The Python version should match your Python version. Also the architecture should match (i.e. 64bits vs 32bits). For example:

  • GDAL-3.1.4-cp38-cp38-win_amd64.whl

This is the GDAL version for Python 3.8 (as can be seen from the cp38 in the name), for 64-bits Python (as derived from the amd64 in the name).

Once you have downloaded the correct files, navigate to the directory in which you saved your downloads. Now type the following commands (the order is important):

  1. pip install GDAL-3.1.4-cp38-cp38-win_amd64.whl
  2. pip install Fiona-1.8.17-cp38-cp38-win_amd64.whl
  3. pip install Shapely-1.7.1-cp38-cp38-win_amd64.whl
  4. pip install python_snappy-0.5.4-cp38-cp38-win_amd64.whl
  5. pip install fastparquet-0.4.1-cp38-cp38-win_amd64.whl

After you've done this you can install hydropandas using pip install hydropandas.

Authors

  • Onno Ebbens, Artesia
  • Ruben Caljé, Artesia
  • Davíd Brakenhoff, Artesia

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hydropandas-0.5.0.tar.gz (76.2 kB view details)

Uploaded Source

Built Distribution

hydropandas-0.5.0-py3-none-any.whl (99.3 kB view details)

Uploaded Python 3

File details

Details for the file hydropandas-0.5.0.tar.gz.

File metadata

  • Download URL: hydropandas-0.5.0.tar.gz
  • Upload date:
  • Size: 76.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for hydropandas-0.5.0.tar.gz
Algorithm Hash digest
SHA256 dd4ee95285c38783f2b3d6d84dca9b27fe8b88af9474475aabff715cba080ac6
MD5 528326753bbb7307bcdecf63d548c057
BLAKE2b-256 fa50d14804042db394e335f6c3a2c3c4f322da04ed93f91ea7c3fe715f14274f

See more details on using hashes here.

File details

Details for the file hydropandas-0.5.0-py3-none-any.whl.

File metadata

  • Download URL: hydropandas-0.5.0-py3-none-any.whl
  • Upload date:
  • Size: 99.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for hydropandas-0.5.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c3a3d2b16c2a975bc5e3c3b52acf2836429d2826742bbafc14a6afea7064cc4f
MD5 04b94a53e08009e4971712544659d52f
BLAKE2b-256 70a6e3df4c0a9490236f600773d52f16849b3572a63c85e847ab9ba7af2eefb4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page