Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.3.0.3-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 50f8584e975e95fd7cc6b48b4995b809007c3219c9d561e1c5a1eb6d10b8dd0d |
|
MD5 | cf02953528ced496e515b634de48f957 |
|
BLAKE2b-256 | b86a652f3e18a7139afee9d3637030b07add313a137d1d5ca19ee516a5f29eb6 |
Hashes for darshan-3.3.0.3-cp39-cp39-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 914565c538869a3a64f18f703a383637f0f3109c2377efb7f049c847c1d49b6a |
|
MD5 | c9e11594336d0af78aef63445422a93a |
|
BLAKE2b-256 | ca5c0427a192dc16ea2b943b91e91300effcd69653fde631d8c2a0ddb89e08de |
Hashes for darshan-3.3.0.3-cp39-cp39-manylinux2010_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dd9b0bdd66657302078e45df4ccb50bdebf08485b076b4b26baff29dbbcc7dbe |
|
MD5 | 0ab1663deeba8d4eb5baed3981edf663 |
|
BLAKE2b-256 | 34b0b5d48d70d1fb76aed4b917eafab36222147f1f9636739e71969cfd3b937b |
Hashes for darshan-3.3.0.3-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa6bea292e400678680c1c03cb9387e793a5747a62e073e2d9a93a66696b139f |
|
MD5 | 48b2a9df59442976d590b9be2f271022 |
|
BLAKE2b-256 | c6af600e472e3f28dea3b7187db23469b5d4c35a882f9480c6e2cadf009ec256 |
Hashes for darshan-3.3.0.3-cp39-cp39-manylinux1_x86_64.manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 46c1b167dff7b93b6ae65769b18ffae3766f331359e08133565b5410a8457885 |
|
MD5 | 85a52dc05b823fc8f461907a562b8e34 |
|
BLAKE2b-256 | b31fbec9e8837eca57c147e7741f640f6f92b4ff087eabc715c3f7579da6c31f |
Hashes for darshan-3.3.0.3-cp39-cp39-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 438517ac8f202d790e792688cae97c134c17db1f9c6bb615f4a84c9e570a39f2 |
|
MD5 | eae3f5b7f78ec4eaeedf93f26ea6a808 |
|
BLAKE2b-256 | 8b4417885c4e98ba5141f49c068d89a5683b9214921d113cc3a2255cd51d4fd0 |
Hashes for darshan-3.3.0.3-cp38-cp38-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6279b21af101d821763b797d1a967563865a592216ff48691c171113fd2c49d2 |
|
MD5 | 369b28da6ce21a7ac51ea41f4a64d00d |
|
BLAKE2b-256 | 767c02ea640a4b739b3916661fb99371b519965ddc27ab55064b68537bf07981 |
Hashes for darshan-3.3.0.3-cp38-cp38-manylinux2010_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 14e51c01a16397d9eda3b33e7f799eb1853c6d6f055e30123ec73c02a8b93d5a |
|
MD5 | e9918540e4ac71eb9008865f5a58edaa |
|
BLAKE2b-256 | c6abd6b8365419602aaa3818774a0d96111ae3dcf89a12e845f34c6e83e47aeb |
Hashes for darshan-3.3.0.3-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5c80a720a7296a9b106d5c7bf345701a727e6dbc70fe77a85588e413f7624637 |
|
MD5 | 707167cb91c4f32ef5a4f7c4d9b6ac76 |
|
BLAKE2b-256 | f9a79f29a6339b3133dccfcf00b7769bc7e932afc5754ca2f2900891ab500750 |
Hashes for darshan-3.3.0.3-cp38-cp38-manylinux1_x86_64.manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0fb6b8954ab1cb20d70c771522e8c8c07e6a527f99e75b6a61e9d69eff266b15 |
|
MD5 | 00b51297cfcdba45e89481e503ada41a |
|
BLAKE2b-256 | 3c37f5f8aff50c3a5269914ff26b57e5c4461190afb72ef0bb5ab0aa80c1c673 |
Hashes for darshan-3.3.0.3-cp38-cp38-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 37f1ac77b18e5692d5134704b476de3d94b4278b797b1b65f335fc33a94a7571 |
|
MD5 | e52167a97e78fcf37a966b23564b76a0 |
|
BLAKE2b-256 | 7a3f959025924515ca6f9fba3986bd9db700b535ee30cb30c28d7015d3d77575 |
Hashes for darshan-3.3.0.3-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f17e745579b4906b4fa19bcaa58d8fd56f4ab83940ec2eba0522b36bdf687a8b |
|
MD5 | 3e04c7bc10c98d7823661aeb8e8536aa |
|
BLAKE2b-256 | 9180fdcd31417f301933cff89aad42f51aa5f33145609fbfbb0e71a389d3c583 |
Hashes for darshan-3.3.0.3-cp37-cp37m-manylinux2010_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f78a93f2bd640af119a6b464205395246ef92c64a824fe2d69f04b1ff4a0b30e |
|
MD5 | 9ca1faabf4136dde054f2645d6067e7b |
|
BLAKE2b-256 | 619de5f0e20f95cf6a3ced7986aca4c0a33754d6f42073fa7f3f819164d07a07 |
Hashes for darshan-3.3.0.3-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d67461a834f3bccb46ad3f88f8f6afb9c4c42afbec90eafde22cb2848c0cc76e |
|
MD5 | 4953a4e74e1a82dcc6396318b24d9e81 |
|
BLAKE2b-256 | 8b7cdb92c4d6c68bcc2e4a36b2f0da9e3312ccdef1952ccb109623e637ad93f6 |
Hashes for darshan-3.3.0.3-cp37-cp37m-manylinux1_x86_64.manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f93454adc0563aff57f47bd234fc618aae608e55d127862699bb13dd841ff17c |
|
MD5 | fa42fc2781bafae200901e0f4af9ca91 |
|
BLAKE2b-256 | b77a8a62b50e25d4c47d0e59b677545db01d212b6bcfcc351d9b13fa49be0d94 |
Hashes for darshan-3.3.0.3-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32896fc062ec9e075693460a1bbf5c8dfb93e660a9c77f354042bef79698222c |
|
MD5 | bc65b3f5ac54abcb9cba5a85f85f7b09 |
|
BLAKE2b-256 | 8a11f0f51f95f9f6cf850035ae5b6d51f9058673bbd26db55577c03d2bc178ca |
Hashes for darshan-3.3.0.3-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa2911539cf689f24c4915cadc5450cc1e8e7defc3f3ba2452d77f61fd46c79f |
|
MD5 | 8d249a2f4b1bb506b49e659fb7691327 |
|
BLAKE2b-256 | fd78e979227b734054c16ccd9bd7c3e63d57fbe7198efbc3f27c10df3a8621ac |
Hashes for darshan-3.3.0.3-cp36-cp36m-manylinux2010_x86_64.manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2384afc78d7291922b08e0c399eeb02d10f9eab46a60e0aae0037ccb1d4a2cf |
|
MD5 | 0055fb0a89e9ca455de1cc242b8005a9 |
|
BLAKE2b-256 | 94b28acea43ba76e599076ebae52f173a612bd0232fa4eb509a5bc631ed1bfa9 |
Hashes for darshan-3.3.0.3-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0707af67596450c62e5ea732e0249aca08f82178d043920400d972478a68b88d |
|
MD5 | 5deef3ff459832825a3a6bcebdfc9c63 |
|
BLAKE2b-256 | 973431caa920a9c4f3a097e22ea9b2aca40221e80a47f49568dabb0eff7d2c27 |
Hashes for darshan-3.3.0.3-cp36-cp36m-manylinux1_x86_64.manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d1b17ffc5f21fee9b423f8d3534245c644c368f8516637c804e3658e4e4b49a5 |
|
MD5 | a2149a4e2f4730cf922f4826ea00b07e |
|
BLAKE2b-256 | 7d7775450ed2947fbeb21bbb97c1fea693692a53a16240d1aa4e447df63f4db5 |
Hashes for darshan-3.3.0.3-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b1a5d01f83a86a0b97131c5dfa2ad0357e33ad6650a682a8f6d5b629663bf16 |
|
MD5 | cca29d95473a8973fb850ea16962d7c3 |
|
BLAKE2b-256 | 5615496bd62571bf7d4a6e611fba84d3fc601e85ba2e3803f3b094d0a557b344 |