Skip to main content

Parse soil measurument data.

Project description

PYGEF

Code style: black PyPi Version GitHub stars Coverage Status

Simple parser for *.gef files. These are ASCII based files used for soil properties measurements. Compatible with Python 3.9.

Recently added the parsing of xml boreholes file, the xml parsing is still in a preliminary phase, not all the files are supported. If you find a file that doesn't work with pygef, please make an issue about it or PR :)

Installation

Latest stable version:

$ pip install pygef

Cutting-edge version (might break):

$ pip install git+https://github.com/cemsbv/pygef.git

CPT files

>> > from pygef import read_cpt
>> >  # read gef and xml files
>> > cpt_data = read_cpt("./my-cpt.xml")
>> > cpt_data
CPTData: {'bro_id': 'CPT000000099543',
          'cone_diameter': 44,
          'cone_surface_area': 1500,
          'cone_surface_quotient': 0.67,
          'cone_to_friction_sleeve_distance': 100,
          'cone_to_friction_sleeve_surface_area': 22530,
          'cone_to_friction_sleeve_surface_quotient': 1.0,
          ...
              'zlm_pore_pressure_u3_after': None,
'zlm_pore_pressure_u3_before': None}
>> >  # access the underlying polars DataFrame under the `data` attribute
>> > cpt_data.data.head()
shape: (5, 9)
┌────────────┬───────┬───────────┬────────────┬─────┬────────────┬────────────┬────────────┬────────────┐
 penetratio  depth  elapsedTi  coneResist  ...  inclinatio  inclinatio  localFrict  frictionRa 
 nLength     ---    me         ance             nNS         nResultant  ion         tio        
 ---         f64    ---        ---              ---         ---         ---         ---        
 f64                f64        f64              i64         i64         f64         f64        
╞════════════╪═══════╪═══════════╪════════════╪═════╪════════════╪════════════╪════════════╪════════════╡
 0.0         0.0    -9.99999
e  -9.99999e5  ...  -999999     -999999     -9.99999e5  -9.99999e5 
                    5                                                                          
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┤
 0.02        0.02   11.0       2.708       ...  0           0           0.03        0.6        
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┤
 0.04        0.039  13.0       4.29        ...  0           0           0.039       0.8        
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┤
 0.06        0.059  15.0       5.124       ...  0           0           0.045       0.9        
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┤
 0.08        0.079  17.0       5.45        ...  0           0           0.049       1.0        
└────────────┴───────┴───────────┴────────────┴─────┴────────────┴────────────┴────────────┴────────────┘

Bore files

>> > from pygef import read_bore
>> >  # read gef and xml files
>> > bore_data = read_bore("./my-bore.xml")
>> > bore_data
BoreData: {'bore_hole_completed': True,
           'bore_rock_reached': False,
           'data': (13, 8),
           'delivered_location': Location(srs_name='urn:ogc:def:crs:EPSG::28992', x=158322.139, y=444864.706),
           'delivered_vertical_position_datum': 'nap',
           'delivered_vertical_position_offset': 10.773,
           'delivered_vertical_position_reference_point': 'maaiveld',
           'description_procedure': 'ISO14688d1v2019c2020',
           'final_bore_depth': 12.0,
           'final_sample_depth': 12.0,
           'research_report_date': datetime.date(2021, 10, 19)}
>> >  # access the underlying polars DataFrame under the `data` attribute
>> > bore_data.data.head()
shape: (5, 8)
┌────────────┬────────────┬────────────┬──────────┬────────────┬────────────┬────────────┬─────────┐
 upper_boun  lower_boun  geotechnic  color     dispersed_  organic_ma  sand_media  soil_di 
 dary        dary        al_soil_na  ---       inhomogeni  tter_conte  n_class     st      
 ---         ---         me          str       ty          nt_class    ---         ---     
 f64         f64         ---                   ---         ---         str         list[f6 
                         str                   bool        str                     4]      
╞════════════╪════════════╪════════════╪══════════╪════════════╪════════════╪════════════╪═════════╡
 0.0         1.0         zwakGrindi  donkergr  false       nietOrgani  middelgrof  [0.2,   
                         gZand       ijs                   sch         420
tot630u  0.0,    
                                                                       m           ...     
                                                                                   0.0]    
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┤
 1.0         1.1         zwakGrindi  donkergr  false       nietOrgani  middelgrof  [0.2,   
                         gZand       ijs                   sch         420
tot630u  0.0,    
                                                                       m           ...     
                                                                                   0.0]    
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┤
 1.1         2.0         zwakZandig  standaar  false       nietOrgani  null        [0.0,   
                         eKleiMetGr  dBruin                sch                     0.1,    
                         ind                                                       ...     
                                                                                   0.0]    
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┤
 2.0         3.0         zwakZandig  standaar  false       nietOrgani  null        [0.0,   
                         eKlei       dGrijs                sch                     0.0,    
                                                                                   ...     
                                                                                   0.0]    
├╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┤
 3.0         4.0         zwakZandig  donkergr  false       nietOrgani  null        [0.0,   
                         eKlei       ijs                   sch                     0.0,    
                                                                                   ...     
                                                                                   0.0]    
└────────────┴────────────┴────────────┴──────────┴────────────┴────────────┴────────────┴─────────┘

Plotting

from pygef import read_cpt, read_bore
from pygef.plotting import plot_cpt, plot_bore

# plot cpt file
plot_cpt(read_cpt("myfile.xml"))

# plot a bore file
plot_bore(read_bore("myfile.xml"))

Documentation

Build the docs:

python -m pip install --upgrade pip setuptools
pip install -r requirements.txt
pip install .

sphinx-build -b html docs public

Format

We format our code with black and isort.

black --config "pyproject.toml" .
isort --settings-path "pyproject.toml" .

Lint

To maintain code quality we use the GitHub super-linter.

To run the linters locally, run the following bash script from the root directory:

docker run \
--env VALIDATE_ALL_CODEBASE=false \
--env RUN_LOCAL=true \
--env VALIDATE_JSCPD=false \
--env VALIDATE_CSS=false \
--env VALIDATE_BASH=false \
--env VALIDATE_YAML=false \
--env VALIDATE_PYTHON_PYLINT=false \
--env VALIDATE_NATURAL_LANGUAGE=false \
--env VALIDATE_MARKDOWN=false \
--env LINTER_RULES_PATH=/ \
--env PYTHON_BLACK_CONFIG_FILE=pyproject.toml \
--env PYTHON_ISORT_CONFIG_FILE=pyproject.toml \
--env PYTHON_MYPY_CONFIG_FILE=pyproject.toml \
--env PYTHON_FLAKE8_CONFIG_FILE=.flake8 \
-v $(pwd):/tmp/lint ghcr.io/super-linter/super-linter:v6

UnitTest

Test the software with the use of coverage:

python -m pip install --upgrade pip setuptools
pip install -r requirements.txt
pip install .

coverage run -m pytest

Requirements

Requirements are autogenerated by pip-compile with python 3.9

pip-compile --extra=test --extra=docs --extra=lint --extra=map --output-file=requirements.txt pyproject.toml

To update the requirements within the defined ranges, run:

pip-compile --upgrade --extra=test --extra=docs --extra=lint --extra=map --output-file=requirements.txt pyproject.toml

Automatically upgrade your Polars code so it's compatible with future versions.

pip install -U polars-upgrade
polars-upgrade src/pygef tests --target-version=0.19.19

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pygef-0.11.0.tar.gz (34.1 kB view details)

Uploaded Source

Built Distribution

pygef-0.11.0-py3-none-any.whl (36.0 kB view details)

Uploaded Python 3

File details

Details for the file pygef-0.11.0.tar.gz.

File metadata

  • Download URL: pygef-0.11.0.tar.gz
  • Upload date:
  • Size: 34.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.6

File hashes

Hashes for pygef-0.11.0.tar.gz
Algorithm Hash digest
SHA256 e08658df7ba0a91f650b7b12416a793547a20b9f8b1a180e76b7617d2669bd59
MD5 104c7e67dfcc711ceba881f4df412d83
BLAKE2b-256 8e3d8b5ab9646bf02ee5e70dbb9b2ed6c762538c99dbafeed3a59c0b710dfb56

See more details on using hashes here.

File details

Details for the file pygef-0.11.0-py3-none-any.whl.

File metadata

  • Download URL: pygef-0.11.0-py3-none-any.whl
  • Upload date:
  • Size: 36.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.6

File hashes

Hashes for pygef-0.11.0-py3-none-any.whl
Algorithm Hash digest
SHA256 4ecb91a4543ff4234c00c59fe6008de4efdaa8124ecbf47ab8297371e138b34a
MD5 9e16bbee2b2925cc73fd649b168b533c
BLAKE2b-256 35f73cf4b33ca8f0d31694edf66bada1045e77fec7d28802869e79aa249467c4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page