Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.3.1.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a6485545d5887795bb0e8a71e581859be6434fabd743e146376d0d86bf9594b6 |
|
MD5 | cc574222bb02551525af1c6b95239bc5 |
|
BLAKE2b-256 | e4d039f4539cc82cfbf74a5f83f8e06a778fb2904482c9076932eb6f8153a7ed |
Hashes for darshan-3.3.1.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f92cbb342b431a49256fdfa13a492d4afd8071a53f1db7fde46acb7b008c10ab |
|
MD5 | fd4be79e8660c903da3547a94ad479b5 |
|
BLAKE2b-256 | 566759e0e1185760f819452986797f7584e6ae8d9adefb79645b1edc2a91c49b |
Hashes for darshan-3.3.1.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b599aa3b29be0cd401fcc0d71de149c2e236dd9d67e34525ea0248e317cd3ead |
|
MD5 | 3a4d7e56b4393783a6de27a5ee048c82 |
|
BLAKE2b-256 | 6013d0c9972068fa8768a28e78710db7248fe936c7ecaddcabbadedf5e8dd37f |
Hashes for darshan-3.3.1.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75e8076ec0600ef657bd658270b7709596551fa296d01930a54f5f91a5093dd7 |
|
MD5 | 3f6919848bc8a59e57d38557970663d7 |
|
BLAKE2b-256 | 1bb37f6dc831b5d3f61c12cd71889670cd75512928ac1c67178d2576a513d1ec |
Hashes for darshan-3.3.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dfb43b138de4d05fa4ff96f17ad0054d6cca8062774cfd786886146a5b35a0d3 |
|
MD5 | 71b85ac022a9c1ab9e765c4221ccb553 |
|
BLAKE2b-256 | 32e5b4830727575987d536dcf993f142c2b4ec22650b8623680e7b5224b2f320 |
Hashes for darshan-3.3.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 96334dabefce1dc591f68c563898a9917c42d6fbf8490d1933d23e3a65711855 |
|
MD5 | 05a7bd56e1e90350a89176483efdc8d1 |
|
BLAKE2b-256 | 74c387acd9178728b9d2f0fbc321fa7e4bae81a9e4e76520b2e94293ab42dfb1 |
Hashes for darshan-3.3.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d046667bbdee156d2a231e2169fb9a528bcc03dc5117ed3b9631a9bf58eb7dee |
|
MD5 | 61b424bfb8fde3e9a02e99881c7f5595 |
|
BLAKE2b-256 | 1f788186d3b36fe095e1e8fbeca182ce8f39834ff201bf718be1b97e9a93db84 |
Hashes for darshan-3.3.1.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 08efe48c5231798601f5a4cee43b6544c13e25b67a7a57171389f61a07bb84ab |
|
MD5 | 5f6d10cbc11661947440b947c1548e5c |
|
BLAKE2b-256 | fc56a7d2cd5fc337656c0338e03079b47476e9c09bdd7c8d1409a6303ed337e7 |
Hashes for darshan-3.3.1.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55e1a8981096e98c77578ffd1e305cdbf03ac089a5bf822eed53bcc56dfbb797 |
|
MD5 | 989c0c0b7a1892e57ab92190a15ae5ce |
|
BLAKE2b-256 | e04f4b591f5792f931b30326ffac50794f14dcc966dad4a0bc0e7d6e60481f03 |