Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.4.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 776527c2777cdfcc44c3004cb5b440e4bbc07f9f49483887f49ef84588fe6381 |
|
MD5 | 33d0dcfea17ac00883b543e6dd88f5b5 |
|
BLAKE2b-256 | d3fd6a53abfd8e8f0678d1d57e9abdeb547f93b5d724d0838d6100c3cc982e49 |
Hashes for darshan-3.4.0.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a549587fd53fac415068e9e09ca4aeecd2db6ed4db0842ccd8f7115f91e6a953 |
|
MD5 | 00d629c60a6b959a8241bc525b789094 |
|
BLAKE2b-256 | 9984c1ff5cf332e0e681cdade698d0af651b9f20068737b771042af2d2db965c |
Hashes for darshan-3.4.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d28c4c2dcab8625d05b6d7921198738de3ec3e25c1486d2843f442d197f39da |
|
MD5 | 5dfce1880ae612c2114cfe0f0ec2289c |
|
BLAKE2b-256 | 8d05b012d7b7d3698555d788934bc1643b0ec656398845216a6785994d1faafe |
Hashes for darshan-3.4.0.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 66934e9becd86f7e36dd18e644d9eb2cb996f1bac4ca9568f743b16d78ba7b85 |
|
MD5 | 142b3010510c70b921409af561b9bad9 |
|
BLAKE2b-256 | 5873d57acc0463154d1c874407e472ae80028b11f809bac52d55ed4f447c8f00 |
Hashes for darshan-3.4.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16696275bd68b358a8f422344d33999e44b0756735a14134e3d90dca6a4f3202 |
|
MD5 | 1c3fcb0ffd99eabf84c4756160cb7f17 |
|
BLAKE2b-256 | 12744873a41e15ba61c10b0059e6e8ef0690a76bf5d08044b5673662e22ff5d2 |
Hashes for darshan-3.4.0.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2b0e64d677a8d210e81ba55ec7f6c15e5adc37e2617928d9a381861b88eb4d1 |
|
MD5 | 788d550816bfb227ccb000c1641b7b61 |
|
BLAKE2b-256 | 63b56d72c2e53ec8e1942f140c1e7dd5fbaeaff10ac01d2d1377fcc321eee54d |
Hashes for darshan-3.4.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2a020e7916757d4733d8774a126effefa4b4f5a5aa92dbbe1dc48dae169e643b |
|
MD5 | f71639d2e48d4e3ae5891d8ef38b0554 |
|
BLAKE2b-256 | 375b2f5b79e7bd34c70f399f3059a2b244b72609258a7f4cd30b06beec79a2a1 |
Hashes for darshan-3.4.0.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 929f220d54564b160d694bc4aa7a553b57e269e573a77d77006a513957a43504 |
|
MD5 | fb9d09ce5ce49e9dc6505ee742627c23 |
|
BLAKE2b-256 | 20304968f00ab34105407f4882ebb0c94c37d4f658e156cd234cd4c9cabedf43 |
Hashes for darshan-3.4.0.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ccdfa321af397b2d466b32794da51cb31cd44e56756d5f7ed4fbc5fd26053d3e |
|
MD5 | 9cc72275811a5b33ac2bc80ac3e5cda3 |
|
BLAKE2b-256 | d911645c6e6333fc5862edba31ee7974dceaa61b3d1f8009afab44ba9ad5207b |
Hashes for darshan-3.4.0.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 28122f04752f6481dbc20ff41c87d76f4fa1f06080606bb8ff61a34439f6a66f |
|
MD5 | 26e1bf41fadd4febfaf6c47f9452f680 |
|
BLAKE2b-256 | 055f35eea69a1c9f2cf7c94cfeadf589d73d102447f5eb1b5ee5fb060258bde2 |