Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.4.0.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b6711fc5e715cc2270b0ed26616360f10b98b90ccc6982f0a017728b86fa3ce6 |
|
MD5 | 92c7373fe4f3899363cbc89f823a86b5 |
|
BLAKE2b-256 | 359f927d393e5e0063aa6cc67734a61da7330b95f98c070bbab4dbc946a08d96 |
Hashes for darshan-3.4.0.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8baf5dd6d5306a11325c4497d76e9414253e26dbfd834546a3e09812366936da |
|
MD5 | 8049a1a35f576837db657e5a9c77d590 |
|
BLAKE2b-256 | fb1e66bdb960ca638b823b3b05bb7396076a7650c5dd9238bbbac796a43683d8 |
Hashes for darshan-3.4.0.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64784f12ce82451a6538c986389de1fc5ab02b6c4569c14c28b84427dd9d7715 |
|
MD5 | 9b9120543da9c71b133f1132439fceb6 |
|
BLAKE2b-256 | 1e486b981a218615874eec25c2474fd32b012b95e542077bd227a8aa1065929b |
Hashes for darshan-3.4.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 96bd089fb1dcb3ea9bfe79a6f3d38a344ed31af0802c57735b1afe024f617e9d |
|
MD5 | 431c58b4c5149a45ab19119ca360dee5 |
|
BLAKE2b-256 | b86406905652de9baddd62ea7c78bc2d1156648d14c62ddfca0532564d5bb428 |
Hashes for darshan-3.4.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9849c728b27e34426c31a74181e9a36b269c79424574a7b31ff5d58cdc8dd902 |
|
MD5 | a89288f64673eb19be7d1ee7363e0d9e |
|
BLAKE2b-256 | 3c54f7760ede039f043f00fd6b723c4838cf3f593d888c54e2dbb7df1d029501 |
Hashes for darshan-3.4.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 20fc2e82feff20375af644564866405a2f3fdfb0ca942991876ec0371aeb0215 |
|
MD5 | 5f525d40d718907bbf390d20dd054db9 |
|
BLAKE2b-256 | 4566ed28e10e7b8fa5c0f7a789ddc81225050d384dabf393615bf04cc882957f |
Hashes for darshan-3.4.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 90a430cca1aceeebfd9463dc711b1f1803161233b4ea930a79d5cef2ec418484 |
|
MD5 | e63163e92dc0247d5fd9201dcf5c2985 |
|
BLAKE2b-256 | 34e431717c1f6b61474a2b85e733d62160b2a17f7b39200610447f658ffc8d7d |
Hashes for darshan-3.4.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 882ac613b6f3a11a993e49dad35cbfa4a714d70ca43c2e1041306d6b24af2bb5 |
|
MD5 | 6bab2e0ff8ae8b24e881ecb1a2f83622 |
|
BLAKE2b-256 | e7f5360f448f334b55e47fc06d664122cd7e6fbb05bf3c6c258e9dad0631b11d |