Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.3.0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f5b651dae7712132b3054929d98f0e8aab19ac099794b85bd67d3266ba9097a |
|
MD5 | 88e256854c79019d143e95491e2b4a10 |
|
BLAKE2b-256 | 1ab6539b7ce410062ab0ebc51743d61d16f03b5669ac70f0ab702f386c497fe4 |
Hashes for darshan-3.3.0.1-cp39-cp39-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8f2f663ac87c54023d319c11008df23bfde1eafb77fc6252b891635cc1366f1 |
|
MD5 | e3adc227df02141a8c8d18fb958b2eda |
|
BLAKE2b-256 | c1bd5db8754e1b99ef1ebbea2f6eaa63c5b1e66de5b049d6f7f9b31f13bfb177 |
Hashes for darshan-3.3.0.1-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 58647247879d575507c5f9c77db0bc292b90e30024c31fcb4b8849596fbb85a4 |
|
MD5 | b428731d4458aee706eef157b285426c |
|
BLAKE2b-256 | 2cfab05fe3691b4b44df50e06a3246ce7ba7794d05c6c54c5da14895deccd3f4 |
Hashes for darshan-3.3.0.1-cp39-cp39-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 27b63ac16d75d7ebd752f5fc96d230f0528375a9f3c3a2f55221396c76946a26 |
|
MD5 | 9eeca2af8bffbbad7564c80e1a48f95e |
|
BLAKE2b-256 | 516c16dd41cf7d5f874bccb32f9ce3676e2927a0a22b50f9d9f5b8862edf81e1 |
Hashes for darshan-3.3.0.1-cp38-cp38-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 187f1b4f576da9ea1584b5a3d5c8688e683de47e89c08e967474f018c2247a06 |
|
MD5 | 20df6b4ae1732d313e7cd1f300ea5441 |
|
BLAKE2b-256 | 9abf50308c3dc0733b32aea91f072c8b92cea05933b5f85dc2168972db1330ba |
Hashes for darshan-3.3.0.1-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dfaab296710c2e7519bbb7ba109f4dc5e040d7b684f95c0f77c0d650888432ae |
|
MD5 | 2c62d031879b9b0e14c515ec1765f47d |
|
BLAKE2b-256 | 4855df644dcee34aa14cb8603ecc7f9e9683e5e3f09f5279d2e9753adae2f338 |
Hashes for darshan-3.3.0.1-cp38-cp38-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ddc52f4c96644f66db5d6f70bd700fdf13756667e0f97a819fe8b3357ba004fc |
|
MD5 | 97092217b69b82174781c8f7a2e11b67 |
|
BLAKE2b-256 | b5f62886d123e02196bd49df355aca85774ea64260149bf9896f66e0c85a86e1 |
Hashes for darshan-3.3.0.1-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa7b19bb0f546d08a0f6c475d5395907d6ab47c20f0483a068d548128d2ca59d |
|
MD5 | fc0dacb2b361ec0e3cc7e9baaf74c34b |
|
BLAKE2b-256 | 44354fc7e4cf2eb87b304d4dbb30dc34fafa52b7707b2431156c09d6c3f6d8ca |
Hashes for darshan-3.3.0.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 998e947b14b8bcf14347f6c71db00c7ddabd9aa7bf30466c29ac21754b446884 |
|
MD5 | a932be091b9aa2968ef95c34f47c5a3a |
|
BLAKE2b-256 | 803e1d8f0fcc02936088eafa80a60d5d6878160e9a204edd13ae37000de04007 |
Hashes for darshan-3.3.0.1-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 23f0780420eea8d730637a4b6f8f0f74c4b70f8e4dd8b205a5862a0e528e4c36 |
|
MD5 | 4a6a62f5de163135acf5fa498b9d95dc |
|
BLAKE2b-256 | a25335588a40c2958a1f377d65f7ef251f031090ec2698f26dfcfc19326d6c18 |
Hashes for darshan-3.3.0.1-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f3c699d9de6ecd66aa78a2c8144f076f40bf9a3a2aee932ef9fbbb009549487 |
|
MD5 | ecd45bd7d95ce483d89e89ff8f597cea |
|
BLAKE2b-256 | 7086310748da4b55494605a77cacfbd88c34fceed4810c96820ea8a9fb9cc807 |
Hashes for darshan-3.3.0.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 149f539f8ff28739da59d546d4a1025ceda067e04ac751c34efe3ff9c105c424 |
|
MD5 | fb18cda3867eb06f6aef1940c361a782 |
|
BLAKE2b-256 | ad46ecc8284ed1c4164c95589d931c764d2260531d9a0c047699caa01baf2dd5 |
Hashes for darshan-3.3.0.1-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6a06738310b47e3d48f9cb2773283a1a27a102354cb5e049085e266116a38084 |
|
MD5 | b501f27d2b3e84a90c1b685572050f4a |
|
BLAKE2b-256 | 4b544efddc427bbda502de52c53dc51bd9bff1c2cb3071d37bb0c51a8066ec18 |