Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.4.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 36c6487681b7ae3a969385c3aa7501c555713eba1a7b04c99ef138f324479401 |
|
MD5 | 7e73b016a920bc92799fc757a9c80db1 |
|
BLAKE2b-256 | 0f442e9e2bf49373519448749163c69b16d1c44397eb13a6586102ed385db637 |
Hashes for darshan-3.4.1.0-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a14c3e8cad4426cad9f23ed4e60361f36bab1f218a50110f8d99a53bf7bb958a |
|
MD5 | 77b09c23c0805261a6af0aa36dc95123 |
|
BLAKE2b-256 | 19aef7045f94f4bfc97a1cbe4a29099b5135497ed57b41d752a07b83e2aaf286 |
Hashes for darshan-3.4.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d411922c9963897189dcc7ed529b18e78b52eb59e690447b2210dbb9ea142e9 |
|
MD5 | daa8597fdb3069da0f3e87b972c38c1a |
|
BLAKE2b-256 | 203de81926f57296036bc16de622274ef3a49b2fca8b29d5fa6a09e6ff66a706 |
Hashes for darshan-3.4.1.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16cffd2f577bd0c33633d188972f2c61e08b0c85fa0acdbf854a56bae0f79759 |
|
MD5 | ef7d3f934e64f1f04b86ee36d0ba2628 |
|
BLAKE2b-256 | 5e12fba6c88756ef26c46a5c2704a8a267612575a540c6907046852e0c2d12ea |
Hashes for darshan-3.4.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2bbc5c672352abe00ae9cb3f9318f6ffc0f625430db07446236f12fdb88c4596 |
|
MD5 | 3db52d748b762eeee5bc53e14dd76007 |
|
BLAKE2b-256 | 3cd3f335134a45ae24b38ea7636f9b52c397ad3ebc73c53852c971557f0ed2a7 |
Hashes for darshan-3.4.1.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a2ffd0496e933e148ebd82b66521b24c647a5cbb279a7d1a2ed8adbfe584f993 |
|
MD5 | c1bc98ad7bfa3b060d3d363a34c5d7a5 |
|
BLAKE2b-256 | a6c5b1ffb24f09dab9524befdc4af4e6ab66bb3d1b8ac969378812da9a67c951 |
Hashes for darshan-3.4.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 582d9d5a1279bb546a53e2d5d94081b16ca3b0627e53f233e9ba9cce0188a37a |
|
MD5 | 3a3294f16859798543e1270d2cfcd637 |
|
BLAKE2b-256 | a694b3cfd329492c4534f314748ee4b9b2d97a4d1139f73a1623e44220131409 |
Hashes for darshan-3.4.1.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9d3cc5c396fdf469e8f30675ed22bf84a5105e4074ee2e4e4c6c29f8ae20e2d8 |
|
MD5 | a06779a07861038ca95dcd6f264813aa |
|
BLAKE2b-256 | 0fda8065232f5d7f75566d3c3f57ef9b9b77cc3a578f724ced4018619fa4fc80 |
Hashes for darshan-3.4.1.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35e755403bc75d0e5fbc79baecd0c570c0911db066a9cc2430eb62eb18dd22b7 |
|
MD5 | 2cabb1e3230abd87cfffbf6f7d291f73 |
|
BLAKE2b-256 | a6fb3e1bfea3576b999f50c72a78b6528f30784452d2520b413dff4bc589ba0a |
Hashes for darshan-3.4.1.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3533a8ba8df5a9c5fbfa48d8ebbd81a857c28d7aa6e90774ee27eb873c36116e |
|
MD5 | 6d346d11ac654f2b2740720ad69fed26 |
|
BLAKE2b-256 | 2dbea538241f32701ac9527bbc1c2b747eae6039f2911c0e2d47ec513fd6c11e |
Hashes for darshan-3.4.1.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ede2fb5fe0731464290ea6ad10be64f26737be7704b9ce4daa9f6856badaf54 |
|
MD5 | a8176b89c26f12426ba1807d05472c0c |
|
BLAKE2b-256 | ab51f73e0c8071c2d3a9bce5ed5fb5ff4557ab2101f8b3ac8dd8d663a3f55b40 |
Hashes for darshan-3.4.1.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0bc4ec75b0db842902906133f5aa25c1b9749a0c5f15342150efa8f2448e089a |
|
MD5 | 0a4a654d245726fe1668981be6dadc6d |
|
BLAKE2b-256 | 77aa8190a96d999cd54e89d8190ed05c2c69d3c3e1f943119a09fbf7f8dcdb38 |