Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with PyDarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log with darshan.DarshanReport('example.darshan', read_all=False) as report: # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing PyDarshan usage with log files
- tests::
PyDarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating PyDarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.4.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6a731135d8937b580f132c2ceadc34dff6e05a24143432bd24c7cc4762943119 |
|
MD5 | 947c87bb56f4e28210fa6d8066847c37 |
|
BLAKE2b-256 | 0eeaf11911af110549f18129c09b2606acabeee822594f5832c3cb29d61627af |
Hashes for darshan-3.4.2.0-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1347e344bc6c45c5c7e4a4ca5fbf775ce92d5b61a5aea592b8c67adb79736419 |
|
MD5 | aa500cf7386cdad3dce06c3db8a54ca9 |
|
BLAKE2b-256 | a5a0fb7a2d2b977f8d290fcd6f7a9a940b8fb36cbbf85a8ad17b6a5cad44394a |
Hashes for darshan-3.4.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8cbed76aaa46068a57f0031eb5ca0c9984e8edda67eae005cac81164d348df5 |
|
MD5 | fbcbcf001ee96951e99fd3ce400c3229 |
|
BLAKE2b-256 | c1cd24567e3d22d2dc8170a0f02274cb47e74b0f2d7a615336d914aa5c5e0d9f |
Hashes for darshan-3.4.2.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec5479344617485ecb04f23d4361c55d4c281b04e3328f63beaae646684620cd |
|
MD5 | c07c1c1c12c6d0d88e6a585625180cfd |
|
BLAKE2b-256 | bdc243bcfec3e09b32457b948749c2d49e41925353a46f2f38cc9ff444a58178 |
Hashes for darshan-3.4.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15a6872072c548a824a72dc302debfd62c50d072928f5e53b5fadda38ac1f06f |
|
MD5 | 05756d32437ed73d240e7168fc86ef74 |
|
BLAKE2b-256 | 85b6ed5e05d14a4157659819bf1ab577ca721d841ef8f5ca4536f99f760c354e |
Hashes for darshan-3.4.2.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fff9d9f88f097dfb9f8daeeb836d53cd82ac9a544ca8e747b14ddbde40bc508a |
|
MD5 | 212a6f7e3a8fe2d81a3854c48147c1a3 |
|
BLAKE2b-256 | c54ff5670e9ab96f282807d850de00d03029bea6a1ff124247d3dcd26c653305 |
Hashes for darshan-3.4.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | efbe4de9cd25ca0ad72bdc2e951e556379eb8a9712b7ae1cac834f900f90bce6 |
|
MD5 | b65623fa2ded07f49d1c3503a3ceeee8 |
|
BLAKE2b-256 | 8beae7daf3fc90d38b3e269d408b30d440cbf7ac1741833ae69a285afeadef59 |
Hashes for darshan-3.4.2.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6281e7f8de821b8c37ea2fcaa3b1614eae365278151a78a8a97c88ff2af46d61 |
|
MD5 | 9104c1b266121665e65caea3a89042ac |
|
BLAKE2b-256 | 6cdfa16164f36289bb86c67a77bc2ef43c27b79a2f56d120464b90d97075f926 |
Hashes for darshan-3.4.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | da160e0c34776d5482ea48de10b7644fa7bdfc86c51659c462ab2e9f75d687c5 |
|
MD5 | ad8f0aee4859c02de304df09ef702a80 |
|
BLAKE2b-256 | d763b65a27b78003e682a04f0131babd43a0794656748d2468aedd30815bb44b |
Hashes for darshan-3.4.2.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59b1d6b7130f483a18ccc75d20db7fce01f6db0feabadf757e4f7797a13cee06 |
|
MD5 | d0fd48de90f9451f00ac8b33f2e833a9 |
|
BLAKE2b-256 | fa0183840fb5029f05814937c4be4c7251475f9589008fed2e41fbb0306433cc |
Hashes for darshan-3.4.2.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f6d1f130851aca7491bf064807ac9b28e683f7495d092005b215ce546cd8be39 |
|
MD5 | c1dfdfb3297165f277bea7f74c84b8d4 |
|
BLAKE2b-256 | 5b5a30ddf827d7c25367b185911300be19c0d5d5a0821a23e8f64eff8171cc7b |
Hashes for darshan-3.4.2.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cf70e69f143fc6af302cb899389be995df1651d1671e61d0e01301fc0c08c1a0 |
|
MD5 | 1bb87b42ddfd4eab7a47a8d6dd8d342a |
|
BLAKE2b-256 | 2e82e6b0e29177a5fafceda942b1927646e1aa349518064ae9c0aec9971a8ac9 |