Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.3.0.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e9e5f4565aa5c3dc4dc02c9dc102a9ff3fc939179b0ce057d01e21e872e19b5 |
|
MD5 | da61f3e3d107eb9ad734ad5197b147d3 |
|
BLAKE2b-256 | 3a366573167e10120514837b60f24d32dbff907f5b2ae72a6c132116044f4eaa |
Hashes for darshan-3.3.0.2-cp39-cp39-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d8e44a1e0982b45c15e1b63a0bf618fcc85c1b17f74f7600b289195cc30d886 |
|
MD5 | 03183370ae8dd334885a7ab2cfd41778 |
|
BLAKE2b-256 | 082fad52e7202a56a4512698c5da38dc32ff73daf27c63a9aae89ba983ff19af |
Hashes for darshan-3.3.0.2-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2a83360d4c750dabef46699eeee323cbaafa896f27078ef25926500d58e1fb9e |
|
MD5 | b4b1cb882d7a0b24d95407e759fb9c9b |
|
BLAKE2b-256 | 0ff8d40051d1328990912ca39107c78d7effd24a5a2443d2e896b706673e982e |
Hashes for darshan-3.3.0.2-cp39-cp39-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0ea79603e721751d466fbb5dfa3c8b8419dc8878574161f5e4715ae2a72099db |
|
MD5 | e9af1e1ecb3a734995cafccd944173f6 |
|
BLAKE2b-256 | 4743e79a2f001ff0aad77973dd765e10b9c69a9cda1ba656ab4cf4a4dd3a8e91 |
Hashes for darshan-3.3.0.2-cp38-cp38-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83591ec210d833c353aeec962f7f88ba8e5a472d53472abd17e7a5fe5bf140a9 |
|
MD5 | 4f1efd6d8633a66ba3ddd68fe803bfc9 |
|
BLAKE2b-256 | 8a95eba3604a0813272f252927914e08709ec4c625ebd1c4af1702b31f474425 |
Hashes for darshan-3.3.0.2-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f40e34848b150d3e119ceba75cf48ce26aa7040b6005c341debc77434d3cc159 |
|
MD5 | 6f5c72d07801388b61278e1940bfb95a |
|
BLAKE2b-256 | e88e521cedc5dd44cce7d6cfe9449e97a654890c38ab5b0bd2ed3637cd2a2e4d |
Hashes for darshan-3.3.0.2-cp38-cp38-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6db17f15ca831a1d820d4328ea5c1d1abbce5027d17998de51a63240f23cfc48 |
|
MD5 | ec0bac21a62475d13afb83f26065ee85 |
|
BLAKE2b-256 | 8dd39f2a765edf5e8ac2bbffb4fed5393ffb2e994d2a58aa1e7182cc6af1cfac |
Hashes for darshan-3.3.0.2-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee0f083f110b9aeb2fd2b2c92f7bfafa303c5e20a1a24aba8f66ac5096d113d8 |
|
MD5 | 1dd734a2090364155c9b42c328e049cc |
|
BLAKE2b-256 | bfbf8417122e2266c3e1f3e7fa08c9cca467b33dd83470d46f55c4b36786f4ed |
Hashes for darshan-3.3.0.2-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0ca3450d23658e7b52e47c235afd41b0485696978630a877d36a0a2788d6456f |
|
MD5 | 275e2b5e5ff535ef8147c4c0809f3db4 |
|
BLAKE2b-256 | d242ab024bd6816f4cec0d685f508151656955a355cde4fd1b250fbb26bfb0cd |
Hashes for darshan-3.3.0.2-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d25ac3122c0a47bb432681fbc77758c748412a450efc6341c529b3afab999f07 |
|
MD5 | 5e8d512978b294cd0a2677f7198a32e8 |
|
BLAKE2b-256 | fcb0c616efb1f22cb00c58b7bde5d786146927831c319aa9de91bc65e7f57952 |
Hashes for darshan-3.3.0.2-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eabbaa417ae2e5549f717c94c4464d950efc2183401378327b899e5a88f31cef |
|
MD5 | b61e2f2c0bb4c1b0c8337d4280261595 |
|
BLAKE2b-256 | fab8afe9e862fb04662179fc66d611acf752ad621bf064b844c04f59875c30a9 |
Hashes for darshan-3.3.0.2-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59e7175b230f7c95f4c87bb402da8b1eee8bf2785c81fff0314dcf997cbb1091 |
|
MD5 | 85d32e9359c40295c91d4e0ace2a730a |
|
BLAKE2b-256 | 3c9e01a4ebe16536ff2def7f303d439b453d764229c04eed3a86190f3385f85b |
Hashes for darshan-3.3.0.2-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fe641cac042204254d886003784440a866839b9e91715dadb4b7b2abe684cf45 |
|
MD5 | a5e8be4fe893a4f4e50f7c703a8c1457 |
|
BLAKE2b-256 | 8e227e9438e088e0ba69180c36a3cae146ad0b08515d60bce242ee45bdf60c02 |