A collection of Python tools by Daniel Scheffler.
Project description
py_tools_ds
A collection of Python tools by Daniel Scheffler.
Free software: GNU General Public License v3 or later (GPLv3+)
Documentation: https://danschef.gitext-pages.gfz-potsdam.de/py_tools_ds/doc/
Status
See also the latest coverage report and the nosetests HTML report.
Features
TODO
Installation
py_tools_ds depends on some open source packages which are usually installed without problems by the automatic install routine. However, for some projects, we strongly recommend resolving the dependency before the automatic installer is run. This approach avoids problems with conflicting versions of the same software. Using conda, the recommended approach is:
via conda + pip
# create virtual environment for py_tools_ds, this is optional conda create -y -q -c conda-forge --name py_tools_ds python=3 conda activate py_tools_ds conda install -c conda-forge numpy gdal 'pyproj>=2.1.0' shapely scikit-image pandas
Then install py_tools_ds using the pip installer:
pip install py_tools_ds
via conda channel (currently only for Linux-64)
# create virtual environment for py_tools_ds, this is optional conda create -y -q --name py_tools_ds python=3 conda activate py_tools_ds conda install -c danschef -c conda-forge -c defaults py_tools_ds
Credits
The py_tools_ds package was developed within the context of the GeoMultiSens project funded by the German Federal Ministry of Education and Research (project grant code: 01 IS 14 010 A-C).
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.16.6 (2020-12-10)
Updated installation procedure documentation.
0.16.5 (2020-12-09)
Added URL checker CI job and fixed all dead URLs.
Removed travis related files.
Re-ordered dependencies.
Added Python 3.9 classifier to setup.py and removed those for <3.6.
0.16.4 (2020-12-07)
Fixed issue #15 (boxObj.get_coordArray_MapXY() returns faulty y-coordinate array).
Test_boxObj.test_get_coordArray_MapXY() now also validates the output of boxObj.get_coordArray_MapXY().
0.16.3 (2020-12-03)
Fixed an issue causing geotransform2mapinfo() return a ‘South’ mapinfo instead of a ‘North’ one for a GeoTransform in the West of the central meridian.
0.16.2 (2020-11-18)
Fixed issue of remaining coverage artifacts after running ‘make clean-test.
Revised coord_trafo.py. This fixes an issue that caused pixelToLatLon() to return Lon/Lat instead of Lat/Lon.
Fixed mapXY2imXY() and imXY2mapXY().
Added Test_mapXY2imXY(), Test_imXY2mapXY(), Test_pixelToLatLon(), Test_latLonToPixel().
Removed GDAL dataset input parameters from some functions.
Revised code style and some docstrings. Added some typehints.
Bugfix for always raising a RuntimeWarning in fill_holes_within_poly().
0.16.1 (2020-11-03)
Removed anaconda recipe and deploy_anaconda CI job because py_tools_ds is now on conda-forge anyways.
0.16.0 (2020-11-02)
Removed similarity module as this is only used by arosics (moved SSIM computation there).
0.15.11 (2020-11-02)
Replaced deprecated osgeo imports.
0.15.10 (2020-10-30)
Bumped version due to wrong PyPI upload.
0.15.9 (2020-10-30)
Fixed issue #13 (TypeError when a MultiPolygon instance is passed to fill_holes_within_poly()).
Added tests for fill_holes_within_poly() and get_overlap_polygon()
Tests are now also executable on Windows.
0.15.8 (2020-10-26)
Replaced deprecated ‘source activate’ with ‘conda activate’.
Bugfix for not rounding off negative coordinates in get_smallest_boxImYX_that_contains_boxMapYX() and get_smallest_shapelyImPolyOnGrid_that_contains_shapelyImPoly().
0.15.7 (2020-10-12)
Exclude tests from being installed via ‘pip install’.
Use SPDX license identifier and set all files to GLP3+ to be consistent with license headers in the source files.
0.15.6 (2020-09-15)
Replaced deprecated HTTP links.
0.15.5 (2020-09-02)
Revised build_testsuite_image.sh
Added docker-pull-policy to register command.
Added some comments to build_testsuite_image.sh and set concurrency to 4. Updated deprecated code in .gitlab_ci.yml.
0.15.4 (2020-09-01)
Fixed wrong output of reproject_shapelyGeometry() in case of EPSG 4326.
0.15.3 (2020-09-01)
Fixed wrong output of transform_any_prj() in case of EPSG 4326.
Updated minimal version of pyproj to 2.2.0.
0.15.2 (2020-08-27)
Fixed a race condition within geotransform2mapinfo() and mapinfo2geotransform() when running in multiprocessing.
0.15.1 (2020-08-27)
Fixed exceptions caused by WKT2 strings passed to GDAL<3.
0.15.0 (2020-08-26)
Replaced old code with simple pyproj>2.1 calls. This fixes a PermissionError within WKT2EPSG on Windows.
Added minimal version of pyproj.
Replaced deprecated code in reproject_shapelyGeometry().
geotransform2mapinfo() should now be compatible with gdal<3.
0.14.36 (2020-08-21)
test_py_tools_ds_install now runs within conda-forge environment. Updated installation instructions.
0.14.35 (2020-08-21)
Re-upload to PyPi.
0.14.34 (2020-08-21)
Added ‘tolerance’ keyword to geo.vector.topology.polyVertices_outside_poly() to avoid wrong return values due to float uncertainties.
0.14.33 (2020-08-18)
Removed keywords from gdal_array.OpenArray() call to ensure downwards compatibility.
0.14.32 (2020-08-18)
Fixed DeprecationWarning related to skimage.measure.compare_ssim. Added minimal version of scikit-image.
0.14.31 (2020-08-18)
Moved skimage import to functional level to avoid static TLS import error.
0.14.30 (2020-08-17)
Removed deprecated gdal backports (compatibility module).
Added minimal version of gdal.
Cleaned up some unused code.
0.14.29 (2020-05-19)
Fixed HISTORY.rst and PyPI upload.
0.14.28 (2020-05-19)
Fixed pyproj deprecation warning.
Generated HISTORY.rst.
0.14.27 (2020-03-24)
Merge branch ‘bugfix/fix_issue10’ into ‘master’
Fix for incompatibility of round_shapelyPoly_coords() with shapely>1.6.4.
0.14.26 (2020-03-10)
Fix for returning white spaces at the end of a Proj4 string. Updated version info.
0.14.25 (2020-01-08)
Merge branch ‘enhancement/clean_deps’ into ‘master’
Re-added dependency scikit-image.
The rasterio dependency is now optional.
Added conda update to test_py_tools_ds_install.
Removed not needed dependencies pyqt, scikit-image, lxml and ipython.
0.14.24 (2020-01-07)
Merge branch ‘enhancement/remove_sensormapgeo_code’ into ‘master’
Set conda channel priority to ‘strict’ to avoid using wrong channels.
Fixed broken badge.
Fixed deprecation warning.
Removed all functionality related to transforming sensor to map geometry or vice-versa as this was moved to the new package ‘sensormapgeo’.
Added downloads badge.
0.14.23 (2019-10-17)
Merge branch ‘bugfix/fix_bilinear_sensorgeo_gauss_3D_trafo’ into ‘master’
Fixed issue where SensorMapGeometryTransformer raised an exception when trying to resample a 3D input array.
Fixed SensorMapGeometryTransformer.to_sensor_geometry() not working for resamp_alg=’bilinear’.
The test_reproject module now tests all resampling algorithms instead of only one per test.
0.14.22 (2019-10-16)
Merge branch ‘bugfix/fix_3D_input’ into ‘master’
Fixed incompatibility of SensorMapGeometryTransformer with 3D input arrays. Updated version info.
0.14.21 (2019-08-14)
Replaced deprecated PyPi upload commands by twine. Updated version info.
0.14.20 (2019-07-26)
Merge branch ‘bugfix/fix_importerror_spectral’ into ‘master’
Moved import of ‘spectral’ package to function level.
Merge branch ‘enhancement/add_license_texts’ into ‘master’
0.14.19 (2019-07-22)
Added license notes. Added funding note. Updated version info.
Merge branch ‘bugfix/fix_windows_geometrytrafo’ into ‘master’
0.14.18 (2019-06-14)
Fixed Windows incompatibility of SensorMapGeometryTransformer. danschef 6/13/19, 4:46 PM
Merge branch ‘bugfix/fix_mp_deadlock’ into ‘master’
0.14.17 (2019-03-22)
Restricted mp_alg overriding to nosetest console calls only.
Also include unittest in mp_alg overriding.
Force mp_alg = ‘tiles’ in case SensorMapGeometryTransformer3D is called within a nosetest.
Added parameter ‘mp_alg’.
Merge branch ‘enhancement/share_mp_variables’ into ‘master’
0.14.16 (2019-03-22)
Share large arrays between multiprocessing workers in SensorMapGeometryTransformer3D.
Merge branch ‘bugfix/fix_singleprocessing’ into ‘master’
0.14.15 (2019-03-22)
Fixed bug that caused to use multiprocessing.Pool if number of CPUs is set to 1.
Merge branch ‘bugfix/implement_pyresample_changes’ into ‘master’
0.14.14 (2019-03-21)
Fixed test_reproject module.
Fixed linting.
Removed deprecated install command.
Inherit from gms_base_centos:0.3.
Updated docker container setup.
0.14.13 (2019-03-21)
Declared pyresample as conda dependency.
0.14.12 (2019-02-19)
Replaced get_area_def by AreaDefinition.
Implemented pyresample changes and added minimal version of pyresample.
Fixed issue #7 (only appearing on Windows).
Merge branch ‘bugfix/fix_static_TLS_import_error’ into ‘master’
0.14.11 (2019-02-19)
Cleaned up __init__.py files.
Moved rasterio imports to function level to avoid static TLS conflict with pykdtree which is a dependency of pyresample.
Merge branch ‘bugfix/fix_ETRS_projection_incompatibility’ into ‘master’
0.14.10 (2019-02-19)
Reverted pyresample imports. Fixed style issues.
Added workaround for ETRS/LAEA projection incompatibility + tests.
Moved all pyresample imports to the top of the module to avoid Exception: dlopen: cannot load any more object with static TLS.
Moved all pyresample imports to methods to avoid Exception: dlopen: cannot load any more object with static TLS.
Fix for Exception: dlopen: cannot load any more object with static TLS.
Fixed multiprocessing issue related to OpenMP multiprocessing within pykdtree as called by pyresample. Fixed type hints.
0.14.9 (2019-02-14)
Fixed FutureWarning for numpy>1.16: arrays to stack must be passed as a “sequence” type such as list or tuple. Updated version info.
Renamed test method.
0.14.8 (2019-01-21)
Fixed gdal v2.4.0 incompatibility and added tests for io.raster.gdal.
0.14.7 (2019-01-09)
Merged enhancement/add_3D_geolayer_support into master.
Fixed duplicate return value.
Fixed sub-multiprocessing error.
Added SensorMapGeometryTransformer3D + tests.
Typo fix.
0.14.6 (2018-12-14)
Fixed faulty output validation.
0.14.5 (2018-12-14)
Fixed wrong assertion.
0.14.4 (2018-12-14)
Added default to SensorMapGeometryTransformer.to_map_geometry. Updated version info.
0.14.3 (2018-12-14)
Revised SensorMapGeometryTransformer + tests.
Improved output validation.
0.14.2 (2018-12-13)
Fix.
Added additional stage to fix deployment order.
Fixed bad .gitlab-ci.yaml.
0.14.1 (2018-12-13)
Added type hints. Changed a default value. Fixed deployment order to PyPi, Anaconda.
0.14.0 (2018-12-12)
Missing test datasets are now versioned.
Revised SensorMapGeometryTransformer (now fully operable) and added corresponding tests.
Added boxObj.buffer_mapXY() + test.
Added type hints.
Enhanced documentation.
Fixed docker test run.
Added resampling algorithms ‘bilinear’ and ‘custom’. Added docstrings and type hints.
Added first working version of SensorMapGeometryTransformer.
0.13.7 (2018-12-03)
Added a tolerance to get_smallest_boxImYX_that_contains_boxMapYX() to avoid float coordinate rounding issues.
0.13.6 (2018-12-03)
Added tests for geo.vector.geometry module. Bugfixes for boxObj().
0.13.5 (2018-12-03)
Fixed a wrongly raised warning within warp_ndarray(). Updated version info.
0.13.4 (2018-12-03)
Bugfix for find_nearest(). Updated version info.
0.13.3 (2018-12-03)
Fixed linting.
0.13.2 (2018-12-03)
Added version file.
Fixed linting.
Added tolerance parameter to find_nearest().
Added tests for find_nearest().
Docker CI image now inherits from gms_base_centos:0.2.
CI setup now updates ci_env environment installed via docker_pyenvs instead of creating an independent environment.
CI Python environment is not separate from the base env. Fixed mixed channels for gdal and libgdal causing libkea issue during CI.
Capped version of pycodestyle to <2.4.0 due to bug if used together with flake8.
Added kealib to requirementsand to test_py_tools_ds_install to fix import error after install.
Updated docker runner build script.
0.13.1 (2017-12-06)
Merge branch ‘bugfix/fix_decompress’
0.13.0 (2017-12-06)
Updated version info.
Updated README.
Updated .gitlab-ci.yml.
Updated .gitlab-ci.yml.
Updated .gitlab-ci.yml. Updated README.
Updated .gitlab-ci.yml. Updated README.
Revised compression.decompress.decompress.
Added missing anaconda-client. Added conda_build_config.yaml
Bugfix.
Added conda recipe. Removed superfluous packages from environment*.yml. Updated .gitlab-ci.yml.
Updated docker installer and environment*.yml
0.13.0 (2017-12-06)
Added SQL db tools.
0.12.5 (2017-11-30)
Removed buggy assertion.
0.12.4 (2017-11-20)
Updated version info.
0.12.3 (2017-11-20)
Removed deprecated make rule.
Moved docker setups for basic gms dependencies to external project.
Removed additional env ‘py3’.
0.12.2 (2017-11-18)
Beautified docker installer workflow.
Updated env settings within gitlab_ci.yml
Changed tag.
Changed used environment within gitlab CI.
Changed basic image name for py_tools_ds_ci.docker.
Revised docker installer (now uses a basic conda environment and the gitlab runner container on top).
Moved geopandas to pip packages within docker container setup.
0.12.1 (2017-11-16)
Replaced pandas by geopandas within CI installer test.
0.12.0 (2017-11-16)
Added spectral to dependencies. Added many functions from arosics:
Added comment.
Updated pip requirements.
Added badges. Updated pip requirements.
Added badges.
0.11.1 (2017-11-15)
Updated version info.
0.11.0 (2017-11-15)
Merge branch ‘feature/add_rotation_support’
Removed print statement.
Completed implementation of geo.map_info.Geocoding class. Added tests for rotated datasets.
Added support for rotated datasets: Implemented class geo.map_info.Geocoding(). Reimplemented geotransform2mapinfo()
and mapinfo2geotransform(). Bugfix for geo.projection.isLocal()
0.10.1 (2017-11-09)
Progressbar now prints to sys.stderr to avoid conflicts with sys.stdout.
Added FIXME.
0.10.0 (2017-11-02)
Merge branch ‘feature/add_get_array_tilebounds’
Revised get_array_tilebounds() and added tests.
0.9.4 (2017-11-02)
Allowed tuple objects to be passed to warp_ndarray().
Fixed warp_ndarray() in case a list of ndarrays is provided.
Added function numeric.array.get_array_tilebounds + tests.
Added function numeric.array.get_array_tilebounds + tests.
Added requirements_pip.txt.
Added pandas to packages installed by conda during CI.
0.9.3 (2017-10-12)
Bugfix warp_ndarray.
Changed downsampling threshold of geo.raster.conversion.raster2polygon.
0.9.1 (2017-10-11)
Fixed pages.
Updated ‘pages’ CI job.
Renamed CI job ‘deploy_pages’ tp ‘pages’.
Changed deploy_pages CI job to make pages work again.
Updated Anaconda version within docker setup. Updated runner version. Revised .gitlab-ci.yml.
Changed warp_ndarray projection defaults. Added geo.projection.isLocal(). Added module numeric.numbers. Updated version info.
0.9.0 (2017-10-09)
Merge remote-tracking branch ‘remotes/origin/feature/add_localCS_compatibility’
0.8.4 (2017-10-06)
Updated Test_move_shapelyPoly_to_image_grid.
mapinfo2geotransform(): Fix for asserting a map_info with 8 elements in case of arbitrary coordinates. Added test_coord_grid module.
0.8.3 (2017-10-06)
geotransform2mapinfo, mapinfo2geotransform: added compatibility to local coordinate systems. Added test_map_info module.
Added badge for Anaconda cloud.
0.8.2 (2017-09-25)
Fixed issue#3 (typing).
0.8.1 (2017-09-22)
Removed tarfile and zipfile from requirements as they are system libs. Updated version info.
0.8.0 (2017-09-22)
Added module ‘compression’. Updated requirements.
0.7.4 (2017-09-20)
geo.raster.raster2polygon(): Added auto-downscaling of input array and updated version info.
0.7.3 (2017-09-20)
Suppressed inspection.
Fix mapinfo2geotransform for asserting wrong length of map info in case of geographic coordinates.
0.7.2 (2017-12-19)
Added type hint.
PEP8 editing. Added linting.
0.7.1 (2017-09-13)
Merge branch ‘bugfix/fix_get_overlap_polygon’
Merge branch ‘enhancement/add_auto_setter_GDAL_DATA’
Fix issue #5 (get_overlap_polygon() did not return geometry type ‘Polygon’ but GeometryCollection.”).
0.7.0 (2017-09-17)
Fix nosetests.
Added link for nosetests HTML reports to README.rst.
Added nosetests. Activated test test artifacts for failed pipelines. Added test_py_tools_ds_install.
Updated docker container setup and test requirements.
0.6.0 (2017-09-11)
Added environment module containing auto-setter for GDAL_DATA variable.
0.5.0 (2017-09-11)
Added os compatibility module. Added future imports to ensure Python 2.7 compatibility.
Updated README.rst.
0.4.6 (2017-09-11)
Fixed EPSG2WKT returning None in case GDAL_DATA environment variable is not set. Added Test_EPSG2WKT.
0.4.5 (2017-09-11)
Bugfix prj_equal: Removed superfluous projection comparison, improved type hint.
0.4.4 (2017-09-09)
Revised geo.projection.WKT2EPSG and added _find_epsgfile() to increase operation system compatibility.
Added module test_projection. Updated version info.
Added dummy code for conda deployment for other Python versions.
Removed logout.
Added token.
Added another logout.
Always log out from anaconda.
Fix.
Install patch.
Fix.
Fix.
Fix.
Changed conda skeleton output dir.
Fix.
First setup for Anaconda CD.
0.4.3 (2017-08-20)
Fixed missing dependency for scikit-image.
0.4.2 (2017-08-19)
Completely excluded geoarray from py_tools_ds (solves circular dependency).
0.4.1 (2017-07-05)
updated __version__ and __versionalias__
0.4.0 (2017-07-03)
Added auto-deploy to PyPI; revised badges.
0.3.3 (2017-07-03)
updated setup.py
0.3.2 (2017-07-03)
Updated links within documentation. Updated setup requirements.
Bugfix for SystemError: <built-in function Band_SetNoDataValue> returned a result with an error set
Implemented XY-getters for boxObj.
Changed license to GPL v3.
Updated README.rst
Added requirements.txt
Bugfix
Revision of CI setup,
First setup of CI runner.
Removed osr from setup.py because its included in gdal.
Updated README.
Added subpackages to setup.py
Changed import statements in __init__.py
Changed import statements in __init__.py
0.1.0 (2017-06-09)
First release on PyPI.
Changed module name from ptds to py_tools_ds to fix bug of pip installer. Changed license.
Merged complete package content of py_tools_ds into a cookiecutter package.
Updated a deprecated function call, added new submodule “network”.
moved GeoArray to a new separate library called ‘geoarray’, added convenience module;
20170331_01
added keywords vmin, vmax
some improvements
added keyword to GeoArray.show_map() to make figure zoomable
added functions
GMS_BETA
Bugfix
new keyword for GeoArray.get_mapPos()
Bugfix for GeoArray.metadata.setter
Added output verification for get_overlap_polygon()
Bugfixes
Bugfixes
Bugfix
added progress keyword to GeoArray.get_mapPos() and get_array_at_mapPos()
updated assertion from last commit
added assertion
Bugfix
20170123_01
Bugfix
added array caching: GeoArray and all subclasses now remember the last position read from disk and return it from
memory -> speed improvement
Merge remote-tracking branch ‘origin/master’
geo.coord_grid: - is_coord_grid_equal(): added keyword ‘tolerance’; added type hint and docstring - is_point_on_grid():
added keyword ‘tolerance’; added type hint and updated docstring
geo.coord_grid: - is_coord_grid_equal(): added keyword ‘tolerance’; added type hint and docstring - is_point_on_grid():
added keyword ‘tolerance’; added type hint and updated docstring
20170119_02
revised GeoArray.__getitem__()
geo.vector.geometry: - fixed some broken type hints
20170104_01
geo,map_info: - geotransform2mapinfo(): bugfix
Bugfix
added verbose mode to GeoArray.get_mapPos()
Fix for copied memory address within GeoArray
Bugfix for not silencing GeoArray.reproject_to_new_grid() in quit mode
20161125_01
added function for quickly reprojecting GeoArray to a given pixel grid; mask_nodata is now a subclass of GeoArray
Bugfix for rejecting GeoArray subclasses by GeoArray
20161122_01
added new property ‘mask_baddata’ to GeoArray; some bugfixes and further developments; new dtypes package
simplified GeoArray.__init__(); some bugfixes and further developments
added metadata property to GeoArray
added holoviews visualization for exploring bands-axis in GeoArray
updated __version__
20161112_01
implemented point-wise read processes in GeoArray
20161112_01
some further developments and bug fixes
updated __version__
Bugfix for footprint_poly; revised progress bar handling; bugfix for not resetting timeout start time; some further developments
20161108_01
edited some assertion messages and docstrings, updated __version__
bugfix for raster2polygon
bugfix for missing ‘progress’ attribute of GeoArray, bugfix for note resetting start time in raster2polygon
bugfix for running raster2polygon without timeout
bugfix for overwriting user defined nodata value within GeoArray
added warning if automatic nodata value detection returns unreliable value; bugfix for overwriting user defined attributes of GeoArray
bugfix for error in warp_ndarray assertion; bugfix for wrong array slicing
some bugfixes and further developments
20161102_01
updated __version__
Bugfix for wrong output geotransform within GeoArray._get_plotable_image()
added two functions to GeoArray
fixed a circular import issue
added a lot of feature improvements and further developments
updated GDAL version check, __version__
20161029_01
Bugfix for returning wrong array shape when warping a 3D array
Bugfix for raising a warning although everything is fine
added Python 2.7 compatibilty; added GDAL downwards compatibility
added warpMemoryLimit to warp_ndarray
fixed missing function transform_any_prj() that was caused by a copy paste error
added functions to visualize GeoArray as map; improved GeoArray.show(); some new functions
implemented multiprocessing and GCP based warping into warp_ndarray
modified docstring of calc_FullDataset_corner_positions(); added TODO to boxObj()
some bugfixes and improvements
Bugfix within GeoArray; added functions
geo.raster.reproject: - added a new version of warp_ndarray and renamed the old one to warp_ndarray_OLD: much faster
than the old version and no issues when warping 3D arrays - some modifications to warp_ndarray_OLD -
added get_GDAL_ds_inmem() - added get_GeoArray_from_GDAL_ds() - added warp_GeoArray(): a function to warp GeoArray objects
added a lot of functions, checked importability, refactored map module to ‘geo’
Initial commit
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.