Python tools to interact with darshan log records of HPC applications.
Project description
Python utilities to interact with Darshan log records of HPC applications. PyDarshan requires darshan-utils version 3.3 or higher to be installed.
Features
Darshan Report Object for common interactive analysis tasks
Low-level CFFI bindings for efficient access to darshan log files
Plots typically found in the darshan reports (matplotlib)
Bundled with darshan-utils while allowing site’s darshan-utils to take precedence
Usage
For examples and Jupyter notebooks to get started with pydarshan make sure to check out the examples subdirectory.
A brief examples showing some of the basic functionality is the following:
import darshan # Open darshan log report = darshan.DarshanReport('example.darshan', read_all=False) # Load some report data report.mod_read_all_records('POSIX') report.mod_read_all_records('MPI-IO') # or fetch all report.read_all_generic_records() # ... # Generate summaries for currently loaded data # Note: aggregations are still experimental and have to be activated: darshan.enable_experimental() report.summarize()
Installation
To install in most cases the following will work:
pip install --user darshan
For alternative installation instructions and installation from source refer to <docs/install.rst>
Testing
Targets for various tests are included in the makefile. To run the normal test suite use:
make test
Or to test against different version of Python using Tox:
make test-all
Coverage tests can be performed using:
make coverage
Conformance to PEPs can be tested using flake8 via:
make lint
Documentation
Documentation for the python bindings is generated seperatedly from the darshan-utils C library in the interest of using Sphinx. After installing the developement requirements using pip install -r requirements_dev.txt the documentation can be build using make as follows:
pip install -r requirements_dev.txt make docs
File List
- darshan::
core darshan python module code
- devel::
scripts for building python wheel
- docs::
markdown documentation used by sphinx to auto-generate HTML RTD style doc
- examples::
Jupyter notebooks showing pydarshan usage with log files
- tests::
pydarshan specific test cases
- requirements.txt::
pip requirement file for minimum set of depednencies
- requirements_dev.txt::
pip requirement file for depednencies needed to run development tools
- setup.py::
python file for building/generating pydarshan package
- setup.cfg::
input for setup.py
- MANIFEST.in::
input files for setup.py package
- tox.ini::
input for tox which runs the automated testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for darshan-3.3.0.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 77cde3428c56a7f44db3850110134076bf9e650282c2639c1965deac102d4848 |
|
MD5 | a1fa82d3fa25ff74617fb4097b43443a |
|
BLAKE2b-256 | 6b4e89ef3ec727482c91bc93fdf48b73579ebb754bc63dc3c70f7b50c687e084 |
Hashes for darshan-3.3.0.0-cp39-cp39-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de8f455bd89b94c60b105281665721f0a89af2bff7015ba069f213fbf6deeaaf |
|
MD5 | b27f8dc086a8907f06c6df174857f496 |
|
BLAKE2b-256 | 2c253073df56572249a17a09c99fa9e765e57bc8af6d43b90701e211858b3d03 |
Hashes for darshan-3.3.0.0-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dd06a119ef3ff01ae75df9e8137ea90199115aac122c1cd5a06dc1e6751ac58a |
|
MD5 | 075e2568d8458edd19a26323c18b63c6 |
|
BLAKE2b-256 | 9cf6bc8072ceccb0572e03d794d820d4cf46c3956360d15758d6e5de2bfd86f7 |
Hashes for darshan-3.3.0.0-cp39-cp39-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9a683c1af330640662a87af93d6a8255a62fecbb6e895977d51de428bc6aa46 |
|
MD5 | 6a27375e2cf7d9f0d225cbb184331c6c |
|
BLAKE2b-256 | 3adf97e42a7c48a4e014c4faca2a6189ceb795451c9828c2b05e8b3af1e9654c |
Hashes for darshan-3.3.0.0-cp38-cp38-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d956b086fa1ef9374a9f3fbed768c50ba4b811bbbb7af6d3588c39780626eec8 |
|
MD5 | 04df95a428632e8fce86273047cb8d1a |
|
BLAKE2b-256 | b91b774a268567ed0a7604cb7d14c8818655f80d36284ae8b892ebee7e69856c |
Hashes for darshan-3.3.0.0-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b1fc904278a316265905645f2fab3d6d191cb49bc6beee8c9ea3a34aa25d0756 |
|
MD5 | c7ef5afad9a86b221b7dda576e16d592 |
|
BLAKE2b-256 | 03c1b9ea74fa8b3ed9c40d8d137299e3adb7d16b84cf8a7c3cb7e29721fd0ecf |
Hashes for darshan-3.3.0.0-cp38-cp38-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a2405dc1758462d7798aebbf976e552608e7b22fafeb7c60070e93294eee4e7 |
|
MD5 | 5c8e0b16c200bed8322b33d3f2283439 |
|
BLAKE2b-256 | c1bb2f24ae99eacd80b2ff413c05e628488903b038837a52e6dd61310bf50e1b |
Hashes for darshan-3.3.0.0-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bcd2651552b5ed62909cfc5e20ed63ccdd2cfb66fcd7121a8a53e06a40b904ab |
|
MD5 | 5312b7b4ef1ae40b4fd94233ee00c95c |
|
BLAKE2b-256 | a38b3577fce0f7ee79f35854bdd179f8ec94c00ec0f334873568829c1cf895a5 |
Hashes for darshan-3.3.0.0-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9764619160c43c14796e815860687a3c819ad7b8cd3c11e0238e9fbbff1ac20f |
|
MD5 | 891377fabefc0764c19887e1d303d520 |
|
BLAKE2b-256 | f91ba3b6af1571059a07273586a60519dd6edc02b0af02bf5d714b5ffe1dca82 |
Hashes for darshan-3.3.0.0-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9bba554fad7544dc1d8067b0e563e48dd09a42f991dc306d3aa8207f852a23a3 |
|
MD5 | 5cea1a0849a478d1670056d7ffdd0365 |
|
BLAKE2b-256 | 456bbaba11dc844836e6e98b3b3992b2668f3ba20376be26b5ca21f5c323a506 |
Hashes for darshan-3.3.0.0-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cdf327b47b13ed5dd8cbc3152994f4164bed3d945e9553030adacc573ed83d19 |
|
MD5 | 0c185582da9a26fd29fbd8c1f9512466 |
|
BLAKE2b-256 | 3cbe1130ce97a0f3c82bf9d3d1a9f15f228af3b465289d21b1a2ba57f2e15961 |
Hashes for darshan-3.3.0.0-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69369baf49caade2cb084dffe022fea532cdf8ed2224735d26c0cce7eb972814 |
|
MD5 | 0314227b02dfbfa68209bd999df1981a |
|
BLAKE2b-256 | c07fee1ceae3d965a86ec5dc169568e4650d6d52070cfc01bb7cf47adb4c27bf |
Hashes for darshan-3.3.0.0-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa4c32a0451ddc0144d8b370c823fbf86f4fa8375f8b144312d41494cc266306 |
|
MD5 | 3fadde2b235bb2b86190f791ae9d5060 |
|
BLAKE2b-256 | 51c3ea27cc08892c98adcb0376389f45ceec1dbe561ccf228762d0305e0d3e26 |