ASAM MDF measurement data file parser
Project description
asammdf is a fast parser and editor for ASAM (Association for Standardization of Automation and Measuring Systems) MDF (Measurement Data Format) files.
asammdf supports MDF versions 2 (.dat), 3 (.mdf) and 4 (.mf4).
asammdf works on Python >= 3.8
Status
Continuous Integration | Coveralls | Codacy | ReadTheDocs |
---|---|---|---|
PyPI | conda-forge |
---|---|
Project goals
The main goals for this library are:
- to be faster than the other Python based mdf libraries
- to have clean and easy to understand code base
- to have minimal 3-rd party dependencies
Features
-
create new mdf files from scratch
-
append new channels
-
read unsorted MDF v3 and v4 files
-
read CAN and LIN bus logging files
-
extract CAN and LIN signals from anonymous bus logging measurements
-
filter a subset of channels from original mdf file
-
cut measurement to specified time interval
-
convert to different mdf version
-
export to HDF5, Matlab (v7.3), CSV and parquet
-
merge multiple files sharing the same internal structure
-
read and save mdf version 4.10 files containing zipped data blocks
-
space optimizations for saved files (no duplicated blocks)
-
split large data blocks (configurable size) for mdf version 4
-
full support (read, append, save) for the following map types (multidimensional array channels):
-
mdf version 3 channels with CDBLOCK
-
mdf version 4 structure channel composition
-
mdf version 4 channel arrays with CNTemplate storage and one of the array types:
- 0 - array
- 1 - scaling axis
- 2 - look-up
-
-
add and extract attachments for mdf version 4
-
handle large files (for example merging two fileas, each with 14000 channels and 5GB size, on a RaspberryPi)
-
extract channel data, master channel and extra channel information as Signal objects for unified operations with v3 and v4 files
-
time domain operation using the Signal class
- Pandas data frames are good if all the channels have the same time based
- a measurement will usually have channels from different sources at different rates
- the Signal class facilitates operations with such channels
-
graphical interface to visualize channels and perform operations with the files
Major features not implemented (yet)
-
for version 3
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
-
for version 4
- experimental support for MDF v4.20 column oriented storage
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
- handling of channel hierarchy: channel hierarchy is ignored
- full handling of bus logging measurements: currently only CAN and LIN bus logging are implemented with the ability to get signals defined in the attached CAN/LIN database (.arxml or .dbc). Signals can also be extracted from an anonymous bus logging measurement by providing a CAN or LIN database (.dbc or .arxml)
- handling of unfinished measurements (mdf 4): finalization is attempted when the file is loaded, however the not all the finalization steps are supported
- full support for remaining mdf 4 channel arrays types
- xml schema for MDBLOCK: most metadata stored in the comment blocks will not be available
- full handling of event blocks: events are transferred to the new files (in case of calling methods that return new MDF objects) but no new events can be created
- channels with default X axis: the default X axis is ignored and the channel group's master channel is used
- attachment encryption/decryption using user provided encryption/decryption functions; this is not part of the MDF v4 spec and is only supported by this library
Usage
from asammdf import MDF
mdf = MDF('sample.mdf')
speed = mdf.get('WheelSpeed')
speed.plot()
important_signals = ['WheelSpeed', 'VehicleSpeed', 'VehicleAcceleration']
# get short measurement with a subset of channels from 10s to 12s
short = mdf.filter(important_signals).cut(start=10, stop=12)
# convert to version 4.10 and save to disk
short.convert('4.10').save('important signals.mf4')
# plot some channels from a huge file
efficient = MDF('huge.mf4')
for signal in efficient.select(['Sensor1', 'Voltage3']):
signal.plot()
Check the examples folder for extended usage demo, or the documentation http://asammdf.readthedocs.io/en/master/examples.html
https://canlogger.csselectronics.com/canedge-getting-started/log-file-tools/asammdf-api/
Documentation
http://asammdf.readthedocs.io/en/master
And a nicely written tutorial on the CSS Electronics site
Contributing & Support
Please have a look over the contributing guidelines
If you enjoy this library please consider making a donation to the numpy project or to danielhrisca using liberapay <a href="https://liberapay.com/danielhrisca/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a>
Contributors
Thanks to all who contributed with commits to asammdf:
Installation
asammdf is available on
- github: https://github.com/danielhrisca/asammdf/
- PyPI: https://pypi.org/project/asammdf/
- conda-forge: https://anaconda.org/conda-forge/asammdf
pip install asammdf
# for the GUI
pip install asammdf[gui]
# or for anaconda
conda install -c conda-forge asammdf
In case a wheel is not present for you OS/Python versions and you lack the proper compiler setup to compile the c-extension code, then you can simply copy-paste the package code to your site-packages. In this way the python fallback code will be used instead of the compiled c-extension code.
Dependencies
asammdf uses the following libraries
- numpy : the heart that makes all tick
- numexpr : for algebraic and rational channel conversions
- wheel : for installation in virtual environments
- pandas : for DataFrame export
- canmatrix : to handle CAN/LIN bus logging measurements
- natsort
- lxml : for canmatrix arxml support
- lz4 : to speed up the disk IO performance
- python-dateutil : measurement start time handling
optional dependencies needed for exports
- h5py : for HDF5 export
- hdf5storage : for Matlab v7.3 .mat export
- fastparquet : for parquet export
- scipy: for Matlab v4 and v5 .mat export
other optional dependencies
- PySide6 : for GUI tool
- pyqtgraph : for GUI tool and Signal plotting
- matplotlib : as fallback for Signal plotting
- faust-cchardet : to detect non-standard Unicode encodings
- chardet : to detect non-standard Unicode encodings
- pyqtlet2 : for the GPS window
- isal : for faster zlib compression/decompression
- fsspec : access files stored in the cloud
Benchmarks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for asammdf-7.4.1-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 290bb64e3fc1b8b797fdca0582a8e95ed6ac0fdd32da51f30fd35448fef9e2e7 |
|
MD5 | 71684abb70fff07fa3f81811bc2a768c |
|
BLAKE2b-256 | 00608bcc1ab5e98b2ff949eef315fbeacf148f509ae821d70b88720ec6393883 |
Hashes for asammdf-7.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5dac17f517b721c51a382dbb5e7df7ae06b24b1f6aff04a54c97b466db4cdf87 |
|
MD5 | e1c7ff2614813e4eb5706663aea0e41e |
|
BLAKE2b-256 | 9a216aaa46a87e3288419c83b5b0f321b2ddcc802bc62c0478cc99558d36c599 |
Hashes for asammdf-7.4.1-cp312-cp312-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24cd569beab821b1c6f6c4afab33cf46efb5facca1a3e8a8a348dcea3cacdc1e |
|
MD5 | 1c0af37020bdd3a317620b3af89d151d |
|
BLAKE2b-256 | 4dc3abc68b2371a93a4a7c1d53ba626f39670a514bd654fb7d23ad9944e2ffd2 |
Hashes for asammdf-7.4.1-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 940d2df464b9adaccb87930e5b86e204a408ab4658a09a4d31a257b521c9da8b |
|
MD5 | 6c13ac41db274cc2ab70f7e6834b157b |
|
BLAKE2b-256 | cedc63fb40511be44949092b8b0debade5ef18f847435bde94d57cdf005d0ba8 |
Hashes for asammdf-7.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a3a737f64dd7af92d2cdeca60e64557a6ce69b16f06385d3a3d553eaa5445d25 |
|
MD5 | c931c79814aff7fc32e176973cbe38a5 |
|
BLAKE2b-256 | bf28635cd838931cc7fba5e62d4027a90617491b467d4da2e96604fa2de10d52 |
Hashes for asammdf-7.4.1-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cda29c7a4d9cc93785eb1d1539456c0cae2372689e8cfe44dda2c19fea2a729e |
|
MD5 | b4ef4f92605896147d487ebd7510a87c |
|
BLAKE2b-256 | c40fdb7b70c6ebfbd39f03b3c65d6eebd49e3e781792a72c02db44a842ea9698 |
Hashes for asammdf-7.4.1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f52d54fc508292d468c7fc69592a3211913626b1dbd6905057c6d3b22986cf68 |
|
MD5 | fc5896c8fc6934724b387988d5b3ad37 |
|
BLAKE2b-256 | fe8b595b6963791e94cd053ec89743bba2214b47230254bd62cf31ba1cbb4e7b |
Hashes for asammdf-7.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d05bb0f03f0679943705d21476698f47ef0e89ae26079c75373ea4f2eaf51d36 |
|
MD5 | d50b369f3fe0950a2b87c3cb649e6a78 |
|
BLAKE2b-256 | 539eecf7f72ec7f1347b1ebbb9631e039bb8aee199df9950bc75c034dfc4716d |
Hashes for asammdf-7.4.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ed13374dec1efb69e1724bbce0109002f7fa15ac4dd95c488f9e336589087a35 |
|
MD5 | 0c437122e852b46b51eda2269f435314 |
|
BLAKE2b-256 | 25fb5955dcfe7f656744b3cc51e7cd1cbabd66deb93019e624897f7826b5bd85 |
Hashes for asammdf-7.4.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e1ebda63c37df4cfbf73536c7dabbef137532e746e7e86ccbc4ae6ae9ace32af |
|
MD5 | 04a9b70a5cbbd675bfdf2fce32dab35d |
|
BLAKE2b-256 | dee520145363337b70a198597bda66164232c5cc1406272a6d9bd2071436cc00 |
Hashes for asammdf-7.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 271bbc5e81e0768a6c5a4b084b1016bef874735876ad568669565ab6381bca71 |
|
MD5 | 6a1070fc15ebb805113086f07d3c39b6 |
|
BLAKE2b-256 | d00d13844bd8b59ca75979ba67f355a4aa5d69dd93fdbace177d4e15024461af |
Hashes for asammdf-7.4.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 28362b93d5c938260bedd3c428fa759d2f33fd376fb0b92241f15e30d1b3abbd |
|
MD5 | 7032fa77110cc792394f8da6e9a7a348 |
|
BLAKE2b-256 | b3a50010c6c7d5bf438ae660f6fb9690559c6e370b5b21b2bbcd1f1bf0786a3f |
Hashes for asammdf-7.4.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 42060dfd40e9cc7ec9536d3d5fe5cdc0505e69b9091c482d9e7cc05324c3ed1c |
|
MD5 | 5bfa9cdb864aae163b1e744027e934b4 |
|
BLAKE2b-256 | a4f6e546f46def72ae7cb9587757e2d1a463a37d1537e4ea54f50607e20c99a1 |
Hashes for asammdf-7.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2b29a7b00ac114fd56286e317c2a145512102fe03722e737500fb0ed4be040fc |
|
MD5 | 3dee48d12a412bfc086e65e01265752b |
|
BLAKE2b-256 | afd58e1f7bc4a5dd2eaf99236df49bffe20667fa73be1649e07ec365f300c40d |
Hashes for asammdf-7.4.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c04997e9280a47f6350c43c530c59b409e48d742774edf8390918aab01711b5e |
|
MD5 | 17b3992c4faa38152a59fd4ecb8d8bf9 |
|
BLAKE2b-256 | e1f8449ecf0fe5cf236a8da36df8950f5457bab63c95f4e239f8064852962c94 |