Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-0.0.8.4-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bcf28311cfbef26696b4a148c19cfeccca9153106d90173d8aa92f323350d326 |
|
MD5 | f21fa5202f2d29a3d03365565f1154fb |
|
BLAKE2b-256 | 833f6a204dab89b8ed1051e00d16261cf77d76db2ae6f7b0158281ac17e176e9 |
Hashes for darshan-0.0.8.4-cp39-cp39-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4dda49353c6f2eec663e3db3e627f3a0b4238484c0a41d3e50f330415b5833a7 |
|
MD5 | 8f1a31ca90dbbdf638f43aa121862b69 |
|
BLAKE2b-256 | 3af3bec3d2130233e3a6cc9ad01d2b8cedf4c5207a107037a57fc7068def5bcb |
Hashes for darshan-0.0.8.4-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4b578441dd7a41330372318775868eaa11f5cc001c6ade74693f9b629771c92a |
|
MD5 | 2e662dddb79b7a08952336f395c1dae7 |
|
BLAKE2b-256 | c968655468f7ae74ef5031c1eec3454f374f15e97434bd5f7c92a29ef72d6342 |
Hashes for darshan-0.0.8.4-cp39-cp39-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee2bdf96fd6772e5c73db7f86a75de9292df2a233f223abd8e81aaba0b15879b |
|
MD5 | 21a98954db84020e323927eb1d5e1d13 |
|
BLAKE2b-256 | 94d4cca414d0073e0e1f01792140c81e9b33e681e6c3a3fb121c511ccd9ecfe8 |
Hashes for darshan-0.0.8.4-cp38-cp38-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 49266ebbd281a9ee2a3770cab94b1603434d1f004de059bc819a8959e339a545 |
|
MD5 | 3789c14fe326563a367021711d1958b4 |
|
BLAKE2b-256 | 3230e44c5c289b0f6dc911a7da56263541872675be0426b5d07d37ae085d1881 |
Hashes for darshan-0.0.8.4-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d715c478f2eae0b6191fdb617eac57c78f0272859a433eab0bbd0d518358803 |
|
MD5 | e4d75ea57ae8d6fac34f55afb0c670a0 |
|
BLAKE2b-256 | b85d62f59a39b89bf6603e22897fd8c70393966b6eb9ceefaf0fbf30c93b3a2c |
Hashes for darshan-0.0.8.4-cp38-cp38-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 23d297b11ab16185a9f842ea8c8be6daf7336d263d128c7ce5c30e419b26df39 |
|
MD5 | 3cea6d9743abeeda78df4c6e021bf00b |
|
BLAKE2b-256 | 693f61bd72c22436a52afd058a5130e6833542c119ac845e84492fd752dd4bce |
Hashes for darshan-0.0.8.4-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d3893a9f5e8ccf7c480c029e7a9c0366884a658f83554b8c5b36f964b565395c |
|
MD5 | 3229fd7b83cfe7449ec1d13f5b456f03 |
|
BLAKE2b-256 | 9640cc2a4374f5c3f26355be6d9fbfc8931851ecf69acee7873f810d4ccbff98 |
Hashes for darshan-0.0.8.4-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 925aa117f833bb64b0936e6b3a2d6afa5d92153453a41599840c1a90ffa33689 |
|
MD5 | 5c664a7e97ae2cc36815c5805c71dcd2 |
|
BLAKE2b-256 | cff798249aaf70be8bf6020f6086ff857497eff562ed3071abc044324d66d8b9 |
Hashes for darshan-0.0.8.4-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4699359a331bc922e5d4c8fef2b546b09b331bc9bfaa07d5c4dcb65104427dd3 |
|
MD5 | 281f8eb40ecfe82a48513f7e114d3207 |
|
BLAKE2b-256 | 6099d4b1973017e72896f229660b5fcc7e9aababf4534672d6380596507e6743 |
Hashes for darshan-0.0.8.4-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1516d96b6a02a8612839e3f36e03228906af5a771ec485ba2f8508872e4ae749 |
|
MD5 | 922eab6c058460b2875ef1f177834f83 |
|
BLAKE2b-256 | 8fd0fabf228718517519673434721e4cb7e38d2815f5e08861582299e1ae67dc |
Hashes for darshan-0.0.8.4-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d157003873cf88ea53eb15fc29c0e1756411c9d731a88011568af511ff0bae17 |
|
MD5 | 01a29e1ab74e2ab91b421a97abfabfd6 |
|
BLAKE2b-256 | 1d1c9b74eeb4f204bc36c52c4c27ab92b29046188b6037c99031ae09abc3f100 |
Hashes for darshan-0.0.8.4-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15d58e09fcf5c1ab7a48b218cccb60fe243b39e521ff5cb7ec7ebc85d24c2a85 |
|
MD5 | 067fe6f7a63cbd210a0855ffbd80ad90 |
|
BLAKE2b-256 | d7c9cb8374dc753936350af877955decabfa93785f3dbbb4ac9d091b39236d1f |