Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.3.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cdc049b9c8b8b12cd27d13e9f204dda16dc21ebe5cfcae174947b93c6a8ee2e9 |
|
MD5 | 633160035f7c8a00aca3836aeb1f2725 |
|
BLAKE2b-256 | 63608fa75b0e76963328f8ef6cc2d976e937e2bd541c2a887e3f40cf246080dd |
Hashes for darshan-3.3.1.0-cp39-cp39-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | db826e5415029ab8cf24432ac46f53db7e6884c78c7e384cb862a0d8c54333a4 |
|
MD5 | e5b6733ad6873c509dba72e65e00881d |
|
BLAKE2b-256 | 9c5465a2926717e1e4475ab8ec1b045e1fc8095af0cb8649e59e18aa8d97069a |
Hashes for darshan-3.3.1.0-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9cf66f99dc8078fb22a92ab45c11bd389eece8b56cf7f65caeb310fd8e3a314a |
|
MD5 | 9ad6569b1ca383ee5c9c3e7cf1899d5d |
|
BLAKE2b-256 | 192fc40f688c30df1713aae5a2054336006ffb4aeb44d3602b6d82c6b8a1171a |
Hashes for darshan-3.3.1.0-cp39-cp39-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 153c7fde04f2cf20bbf16fe2756e40257e5b03e94ee43b785d205c0a26c1ead6 |
|
MD5 | bf5b1097da9806ebde96c6a55a3cca53 |
|
BLAKE2b-256 | 0543ee31e7c42114155715bb1f438138d5dd4590cd9a24787a52b3e6b2c24d79 |
Hashes for darshan-3.3.1.0-cp38-cp38-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0809f40f0f104a5e9c8f3a9fc8ea5640483bdf4fa7d158b9932eb9bfa752901d |
|
MD5 | 78bdbaf278902b565f79c215b2966cbd |
|
BLAKE2b-256 | 36c187217634795eb7f5a777ca3764352bcf717679f1bf6521d020899bb4809a |
Hashes for darshan-3.3.1.0-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5333e6e89a22ab5ef344e6cf29c5a2bee2b5606673fcdeb2ed2cf046359ea03e |
|
MD5 | 41a7bcb810b0a4d40facd57bc462778f |
|
BLAKE2b-256 | 0c348b5f2bede09558353c9688f62e85fce62af1dcea7dd08f9f6fef3360fff7 |
Hashes for darshan-3.3.1.0-cp38-cp38-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 96c2cece5c1d4dd49121ef61085135dabfeab7f77b434d02400ca6f7856cfe19 |
|
MD5 | e4ef016d823dfad7c3bb763242dc6497 |
|
BLAKE2b-256 | 11fa393888c65a612ccab37722346d8da3c019c74e8b6bb11c9a8561f7e5f2ce |
Hashes for darshan-3.3.1.0-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f0b459589bdcad86db2ef44e8dcd3558e5a68786267a90a20490e27b30c73a5 |
|
MD5 | e9e89b714f5bc25ad807f38687085734 |
|
BLAKE2b-256 | 8aef1ca20f1ed6932c87628dd5ff6da3f1bd96e87d83c37768d3eca194813514 |
Hashes for darshan-3.3.1.0-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0b54413f6942d62c60d2752cdb9ee3dd7974b932f79f51835b5b59562418388c |
|
MD5 | da2c424d6cc1dbe5b3e1fc4b6cd71234 |
|
BLAKE2b-256 | 8b29b5e9908e8cd76e48288389b7f2e6488b1557b0a0d0c530c12cf8217fa457 |
Hashes for darshan-3.3.1.0-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9a4a4a8698e81b5e87027ba912e24d9c1879e1c4b237dbaeda5229a3bab7dd66 |
|
MD5 | 7ba03a744b5357f7ebf4a08d2a52bd4e |
|
BLAKE2b-256 | b556ee28c8446a3e59e26bfc972633fab3ffdd676aef07f60cdda1dd0df0e144 |
Hashes for darshan-3.3.1.0-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c2a5175378c37b36cefdf288650eb09ed20fd12b73663cfd4041a454c4b2951c |
|
MD5 | aa5debce1c9ff88f70cca40bc72f2f14 |
|
BLAKE2b-256 | c94438532f3c0b3aa9dec8ffc091d8102fcc349b87ef75df1486b833902748f7 |
Hashes for darshan-3.3.1.0-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 65b32c549bc34c5314a990863020775715c30a2ad323ecc66f191c763fa855a6 |
|
MD5 | 3fb787bbc25b49e4716001b2029397f2 |
|
BLAKE2b-256 | fe3d681ffcb6bbe381ec1af5f85ea8814fc020ebae165ac748dafe48146819a8 |
Hashes for darshan-3.3.1.0-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bebf481e01941fa1b01a32ae6cb127416233e0b656de27be3c0cec6920c444b7 |
|
MD5 | 2538f70178bb5875aa04ad88beff0d56 |
|
BLAKE2b-256 | 75218c43c7ceb8b4dcda1b09d2286981421e28d6c8783e32a5fdefadaf187395 |