ASAM MDF measurement data file parser
Project description
asammdf is a fast parser and editor for ASAM (Association for Standardization of Automation and Measuring Systems) MDF (Measurement Data Format) files.
asammdf supports MDF versions 2 (.dat), 3 (.mdf) and 4 (.mf4).
asammdf works on Python >= 3.8
Status
Continuous Integration | Coveralls | Codacy | ReadTheDocs |
---|---|---|---|
PyPI | conda-forge |
---|---|
Project goals
The main goals for this library are:
- to be faster than the other Python based mdf libraries
- to have clean and easy to understand code base
- to have minimal 3-rd party dependencies
Features
-
create new mdf files from scratch
-
append new channels
-
read unsorted MDF v3 and v4 files
-
read CAN and LIN bus logging files
-
extract CAN and LIN signals from anonymous bus logging measurements
-
filter a subset of channels from original mdf file
-
cut measurement to specified time interval
-
convert to different mdf version
-
export to HDF5, Matlab (v7.3), CSV and parquet
-
merge multiple files sharing the same internal structure
-
read and save mdf version 4.10 files containing zipped data blocks
-
space optimizations for saved files (no duplicated blocks)
-
split large data blocks (configurable size) for mdf version 4
-
full support (read, append, save) for the following map types (multidimensional array channels):
-
mdf version 3 channels with CDBLOCK
-
mdf version 4 structure channel composition
-
mdf version 4 channel arrays with CNTemplate storage and one of the array types:
- 0 - array
- 1 - scaling axis
- 2 - look-up
-
-
add and extract attachments for mdf version 4
-
handle large files (for example merging two fileas, each with 14000 channels and 5GB size, on a RaspberryPi)
-
extract channel data, master channel and extra channel information as Signal objects for unified operations with v3 and v4 files
-
time domain operation using the Signal class
- Pandas data frames are good if all the channels have the same time based
- a measurement will usually have channels from different sources at different rates
- the Signal class facilitates operations with such channels
-
graphical interface to visualize channels and perform operations with the files
Major features not implemented (yet)
-
for version 3
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
-
for version 4
- experimental support for MDF v4.20 column oriented storage
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
- handling of channel hierarchy: channel hierarchy is ignored
- full handling of bus logging measurements: currently only CAN and LIN bus logging are implemented with the ability to get signals defined in the attached CAN/LIN database (.arxml or .dbc). Signals can also be extracted from an anonymous bus logging measurement by providing a CAN or LIN database (.dbc or .arxml)
- handling of unfinished measurements (mdf 4): finalization is attempted when the file is loaded, however the not all the finalization steps are supported
- full support for remaining mdf 4 channel arrays types
- xml schema for MDBLOCK: most metadata stored in the comment blocks will not be available
- full handling of event blocks: events are transferred to the new files (in case of calling methods that return new MDF objects) but no new events can be created
- channels with default X axis: the default X axis is ignored and the channel group's master channel is used
- attachment encryption/decryption using user provided encryption/decryption functions; this is not part of the MDF v4 spec and is only supported by this library
Usage
from asammdf import MDF
mdf = MDF('sample.mdf')
speed = mdf.get('WheelSpeed')
speed.plot()
important_signals = ['WheelSpeed', 'VehicleSpeed', 'VehicleAcceleration']
# get short measurement with a subset of channels from 10s to 12s
short = mdf.filter(important_signals).cut(start=10, stop=12)
# convert to version 4.10 and save to disk
short.convert('4.10').save('important signals.mf4')
# plot some channels from a huge file
efficient = MDF('huge.mf4')
for signal in efficient.select(['Sensor1', 'Voltage3']):
signal.plot()
Check the examples folder for extended usage demo, or the documentation http://asammdf.readthedocs.io/en/master/examples.html
https://canlogger.csselectronics.com/canedge-getting-started/log-file-tools/asammdf-api/
Documentation
http://asammdf.readthedocs.io/en/master
And a nicely written tutorial on the CSS Electronics site
Contributing & Support
Please have a look over the contributing guidelines
If you enjoy this library please consider making a donation to the numpy project or to danielhrisca using liberapay <a href="https://liberapay.com/danielhrisca/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a>
Contributors
Thanks to all who contributed with commits to asammdf:
Installation
asammdf is available on
- github: https://github.com/danielhrisca/asammdf/
- PyPI: https://pypi.org/project/asammdf/
- conda-forge: https://anaconda.org/conda-forge/asammdf
pip install asammdf
# for the GUI
pip install asammdf[gui]
# or for anaconda
conda install -c conda-forge asammdf
In case a wheel is not present for you OS/Python versions and you lack the proper compiler setup to compile the c-extension code, then you can simply copy-paste the package code to your site-packages. In this way the python fallback code will be used instead of the compiled c-extension code.
Dependencies
asammdf uses the following libraries
- numpy : the heart that makes all tick
- numexpr : for algebraic and rational channel conversions
- wheel : for installation in virtual environments
- pandas : for DataFrame export
- canmatrix : to handle CAN/LIN bus logging measurements
- natsort
- lxml : for canmatrix arxml support
- lz4 : to speed up the disk IO performance
- python-dateutil : measurement start time handling
optional dependencies needed for exports
- h5py : for HDF5 export
- hdf5storage : for Matlab v7.3 .mat export
- fastparquet : for parquet export
- scipy: for Matlab v4 and v5 .mat export
other optional dependencies
- PySide6 : for GUI tool
- pyqtgraph : for GUI tool and Signal plotting
- matplotlib : as fallback for Signal plotting
- cChardet : to detect non-standard Unicode encodings
- chardet : to detect non-standard Unicode encodings
- pyqtlet2 : for the GPS window
- isal : for faster zlib compression/decompression
- fsspec : access files stored in the cloud
Benchmarks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for asammdf-7.3.16-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b180fa38cd47a574936810358c4df8c58fc3ae2916d563d50cab6eb14b227dcc |
|
MD5 | 250ed53934af548677e24104407dcc17 |
|
BLAKE2b-256 | 63aec0dd3b449abc818a0cb776431e4bd0f47b0ace400b603305bb9375ef182c |
Hashes for asammdf-7.3.16-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 42ed10b42ada4c0ba88c52784bcf53de89b8780ae6891547df800774ab814274 |
|
MD5 | c9b45efb9f2a3312a5fc5c163a58623e |
|
BLAKE2b-256 | d478728cbf68a32afd9e48726aa564effdf933133ae1cbcc130182b78c192e4d |
Hashes for asammdf-7.3.16-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e85d87348be6eea9addd62c9b729fab3fffbf1ea387058ce795bc58b2af8465c |
|
MD5 | 49ba4850cea9e8b06169b2e7d832291e |
|
BLAKE2b-256 | 3fde4b74a85643e57eb45833c2597d1bfcbdc0bcc85eac7bd4ec425f07db33c8 |
Hashes for asammdf-7.3.16-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5d5b79ae5ac3478d3468d6ff945b0af3bdc78406ecf978c96245468f309a1ef4 |
|
MD5 | 7ecdcead0e6c52d42d3158fe3bf54f7f |
|
BLAKE2b-256 | 132a6f7064a51c3e8851c7b1b75062011945919a5246bfd36ff886f7c42a6180 |
Hashes for asammdf-7.3.16-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b2c98065e5ca2d127c5af4e9fe832ed4722e084867dbaad9dbc5236227fefad |
|
MD5 | a85eb9536246b6f66f603ca3ba697a81 |
|
BLAKE2b-256 | 2c030115b2ad7a0875fa2224e9839d910fd7854ec5164f0017107e60f9e3d4d2 |
Hashes for asammdf-7.3.16-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fcad635e120413338828616b5edb122d85339d3f8dd24e3fd32242ca66880efb |
|
MD5 | 748863bef835f12d74f1705b9f31f40b |
|
BLAKE2b-256 | 46dd8f8a212088684ebc100ac44278507e7f9cbae2ef5832088bc6a8b66de2ca |
Hashes for asammdf-7.3.16-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bab4253f7d42420ddba66b3d285b28579a7e7e6ddac170d9604daceb03d2e1ba |
|
MD5 | cc3f39c5ce7615a0c26ed06340144559 |
|
BLAKE2b-256 | 106b6c1d33e48c96255d6dbf783bba3527b9b69c4ffb8b93cebfcd47d61e02d1 |
Hashes for asammdf-7.3.16-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8dd5e3816986fe8e22bfaf2500521b7c0574a6bc3472d5088a4b57b371924e03 |
|
MD5 | dd041ababa142b7ea743990f87f38951 |
|
BLAKE2b-256 | 52d7cc6fe8115e3620267bfaf7cc76dc25f33284624c35a038083b32042d775d |
Hashes for asammdf-7.3.16-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dc8f25600420920727186096d243d02c40cee782c773d948f629065494c2f540 |
|
MD5 | c6b43bbb8c9668817394b09b62d778ad |
|
BLAKE2b-256 | 9a02a1edbea2eaf2fdd717a663970db954163216333573f26af29aa5cea345e2 |
Hashes for asammdf-7.3.16-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a2f230a590ea74fd71db12b28dee5557557f37a9377d27727b1abef6c1884861 |
|
MD5 | 0de94e92d188681cde2a61d9adeccd9b |
|
BLAKE2b-256 | 1792f7e2b55d84f53758d5b3172204c8ae01a20fb47d9b54f259ec596622bd48 |
Hashes for asammdf-7.3.16-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5be0afc4db73e1969b04fe82b390486ea0b419ed8b1749c36b67dbd7fd11b989 |
|
MD5 | e6d9c68f4ebbb95bdea9c58fe07345e1 |
|
BLAKE2b-256 | 0e51ab65b0c00932bd3f256576c79e8c747a047810a6035fcd74148ab121303f |
Hashes for asammdf-7.3.16-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dea1677d420712d9281738595d883a3b5898ca81efa7a58ba7011c09e6027647 |
|
MD5 | f96a9179c6fbc29b575261a383874761 |
|
BLAKE2b-256 | bcc31cae382a32794d7c41e4fb5aa104e6d3200f7b361e94d9ebc07cdef5bdb6 |
Hashes for asammdf-7.3.16-1-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ccefdfbb70633f62d9fe5805b08d76dfda1eda00edd75dbfa4f6ec53a434d213 |
|
MD5 | d4fc48452b6bc1cbfb9e0a065b363561 |
|
BLAKE2b-256 | f05e2a014fdd045db280d4fca1073b4d0fbca2ae46dde12677ec37a6ba7d4928 |
Hashes for asammdf-7.3.16-1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8a55b6e67dcce5e8afd79047c670021b7cf4454104dff6e6f949201c0186e257 |
|
MD5 | 76aa80d8739e293c58f65ce6305fe9db |
|
BLAKE2b-256 | c4f71c95b844f8a74fa8d1cbdde42e2b02e8ac438422f37bdb30273e553214c5 |
Hashes for asammdf-7.3.16-1-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 22540f7060547121b5c51e4d76c90864249082e0cd82b1f595aa8b4fe6a89a42 |
|
MD5 | 98c0146e0b495bf5c748e71b5e593734 |
|
BLAKE2b-256 | 2fd9a144f62845371aa5584db432714b2dea07036ef1bb4c79f23088c585c421 |
Hashes for asammdf-7.3.16-1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2fa197f6708c237445f930022c97f0e7b4ab80ee2b4aaf7fbef6db6bda2fc4b0 |
|
MD5 | cbc0631341466f2c62258b487876de4f |
|
BLAKE2b-256 | a9bd6af6f59990481e728a1aaece9c391c582738ef28c981668e086795caaefb |
Hashes for asammdf-7.3.16-1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c8cb2644271bd62b9dda61a7aee8bbc3ffd0ea23d0bd706f287c5b0324920709 |
|
MD5 | 4c0260dec072e15a192214da951233b6 |
|
BLAKE2b-256 | 9b60097329f312281a53e768da73880bcdd934af34b5b3a0ba6fa122c0d2df2b |
Hashes for asammdf-7.3.16-1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e7dba8afa7be87425d76114f113dcefafb426700669ea25767025e4d4dc200a1 |
|
MD5 | b7308ad382dbd66166c410001fa729b9 |
|
BLAKE2b-256 | 1a4e4b48283ebd6474fa7ae3c4ca4cc0b67ac5067d741b9ef03a967e70bf9888 |
Hashes for asammdf-7.3.16-1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8502d376ed5fae5d4370d9967d776f03a22b60fb362a6f65cb91ee7237799d1 |
|
MD5 | a1f1f8c5216707f3bb139b1870ccd673 |
|
BLAKE2b-256 | 31c932eb8baf169749b2110d711aebd9107945bf6888b0d5db9a5ff9cd5cb647 |
Hashes for asammdf-7.3.16-1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7213030f8f6b89c53ce410ac47e20aaf97b11e2f555c742d6eda7b729431bfce |
|
MD5 | f11bc2b8b4ddc8dc7f24c420921d1c14 |
|
BLAKE2b-256 | fdbb05827ff10a0591616f60134fff320f4f08f78d37d2b409217199451f9ca1 |
Hashes for asammdf-7.3.16-1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4846450b73557761d9a69282d5950d5fba345f30594acb6663aa0c2dbf3d1f01 |
|
MD5 | ab0d5caec61bcd55d51a92df2a66ec52 |
|
BLAKE2b-256 | e105a5727c159372de3cd027d212328020c16ec2d86692651934ebab2c256757 |
Hashes for asammdf-7.3.16-1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2c800d79e9d790445491281a47b496b72227fb85990e94922f69749f7faa7477 |
|
MD5 | 6ed105f076c54f87518bb4a258cdffbf |
|
BLAKE2b-256 | 72ab21a1fdde95996b8b29eef37f286a8158cc83a2ae7f1b57089fb7087f3fd5 |
Hashes for asammdf-7.3.16-1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a3e5e8abd1435116f63a3dd408b0f9e85d29dbbc9711620b0a1692cbb6ad87e0 |
|
MD5 | c90d2ea4bc031ef8a09f92a6fbb4e881 |
|
BLAKE2b-256 | 45a37ecd121c1c1180742febe55816ddfa72085250a7a62c9f6e2f86ada1abd4 |
Hashes for asammdf-7.3.16-1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4892951e3a22ff8f7fb55f157977199b0fe2f076f825e6fffb674fa4b6c72b44 |
|
MD5 | 95a0085fc6fe6f0afd1bae6bf2de2e9e |
|
BLAKE2b-256 | 9e43dd45ab0af1f02ac9f6ee1ff3cd08a976a5b459a6ac95dc89a12291585162 |