ASAM MDF measurement data file parser
Project description
asammdf is a fast parser and editor for ASAM (Associtation for Standardisation of Automation and Measuring Systems) MDF (Measurement Data Format) files.
asammdf supports MDF versions 2 (.dat), 3 (.mdf) and 4 (.mf4).
asammdf works on Python >= 3.6 (for Python 2.7, 3.4 and 3.5 see the 4.x.y releases)
Status
! | Travis CI | Appveyor | CoverAlls | Codacy | ReadTheDocs |
---|---|---|---|---|---|
master |
PyPI | conda-forge |
---|---|
Project goals
The main goals for this library are:
- to be faster than the other Python based mdf libraries
- to have clean and easy to understand code base
- to have minimal 3-rd party dependencies
Features
-
create new mdf files from scratch
-
append new channels
-
read unsorted MDF v3 and v4 files
-
read CAN and LIN bus logging files
-
extract CAN and LIN signals from anonymous bus logging measurements
-
filter a subset of channels from original mdf file
-
cut measurement to specified time interval
-
convert to different mdf version
-
export to HDF5, Matlab (v4, v5 and v7.3), CSV and parquet
-
merge multiple files sharing the same internal structure
-
read and save mdf version 4.10 files containing zipped data blocks
-
space optimizations for saved files (no duplicated blocks)
-
split large data blocks (configurable size) for mdf version 4
-
full support (read, append, save) for the following map types (multidimensional array channels):
-
mdf version 3 channels with CDBLOCK
-
mdf version 4 structure channel composition
-
mdf version 4 channel arrays with CNTemplate storage and one of the array types:
- 0 - array
- 1 - scaling axis
- 2 - look-up
-
-
add and extract attachments for mdf version 4
-
handle large files (for example merging two fileas, each with 14000 channels and 5GB size, on a RaspberryPi)
-
extract channel data, master channel and extra channel information as Signal objects for unified operations with v3 and v4 files
-
time domain operation using the Signal class
- Pandas data frames are good if all the channels have the same time based
- a measurement will usually have channels from different sources at different rates
- the Signal class facilitates operations with such channels
-
graphical interface to visualize channels and perform operations with the files
Major features not implemented (yet)
-
for version 3
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
-
for version 4
- experiemental support for MDF v4.20 column oriented storage
- functionality related to sample reduction block: the samples reduction blocks are simply ignored
- handling of channel hierarchy: channel hierarchy is ignored
- full handling of bus logging measurements: currently only CAN and LIN bus logging are implemented with the ability to get signals defined in the attached CAN/LIN database (.arxml or .dbc). Signals can also be extracted from an anonymous bus logging measurement by providing a CAN or LIN database (.dbc or .arxml)
- handling of unfinished measurements (mdf 4): finalization is attempted when the file is loaded, however the not all the finalization steps are supported
- full support for remaining mdf 4 channel arrays types
- xml schema for MDBLOCK: most metadata stored in the comment blocks will not be available
- full handling of event blocks: events are transfered to the new files (in case of calling methods that return new MDF objects) but no new events can be created
- channels with default X axis: the defaukt X axis is ignored and the channel group's master channel is used
- attachment encryption/decryption using user provided encryption/decryption functions; this is not part of the MDF v4 spec and is only supported by this library
Usage
from asammdf import MDF
mdf = MDF('sample.mdf')
speed = mdf.get('WheelSpeed')
speed.plot()
important_signals = ['WheelSpeed', 'VehicleSpeed', 'VehicleAcceleration']
# get short measurement with a subset of channels from 10s to 12s
short = mdf.filter(important_signals).cut(start=10, stop=12)
# convert to version 4.10 and save to disk
short.convert('4.10').save('important signals.mf4')
# plot some channels from a huge file
efficient = MDF('huge.mf4')
for signal in efficient.select(['Sensor1', 'Voltage3']):
signal.plot()
Check the examples folder for extended usage demo, or the documentation http://asammdf.readthedocs.io/en/master/examples.html
https://canlogger.csselectronics.com/canedge-getting-started/log-file-tools/asammdf-api/
Documentation
http://asammdf.readthedocs.io/en/master
And a nicely written tutorial on the CSS Electronics site
Contributing & Support
Please have a look over the contributing guidelines
If you enjoy this library please consider making a donation to the numpy project or to danielhrisca using liberapay <a href="https://liberapay.com/danielhrisca/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a>
Contributors
Thanks to all who contributed with commits to asammdf:
- Julien Grave JulienGrv
- Jed Frey jed-frey
- Mihai yahym
- Jack Weinstein jackjweinstein
- Isuru Fernando isuruf
- Felix Kohlgrüber fkohlgrueber
- Stanislav Frolov stanifrolov
- Thomas Kastl kasuteru
- venden venden
- Marat K. kopytjuk
- freakatzz freakatzz
- Martin Falch MartinF
- dxpke dxpke
- Nick James driftregion
- tobiasandorfer tobiasandorfer
Installation
asammdf is available on
- github: https://github.com/danielhrisca/asammdf/
- PyPI: https://pypi.org/project/asammdf/
- conda-forge: https://anaconda.org/conda-forge/asammdf
pip install asammdf
# for the GUI
pip install asammdf[gui]
# or for anaconda
conda install -c conda-forge asammdf
In case a wheel is not present for you OS/Python versions and you lack the proper compiler setup to compile the c-extension code, then you can simply copy-paste the pacakge code to your site-packages. In this way the python fallback code will be used instead of the compiled c-extension code.
Dependencies
asammdf uses the following libraries
- numpy : the heart that makes all tick
- numexpr : for algebraic and rational channel conversions
- wheel : for installation in virtual environments
- pandas : for DataFrame export
- canmatrix : to handle CAN/LIN bus logging measurements
- natsort
- lxml : for canmatrix arxml support
- lz4 : to speed up the disk IO peformance
optional dependencies needed for exports
- h5py : for HDF5 export
- scipy : for Matlab v4 and v5 .mat export
- hdf5storage : for Matlab v7.3 .mat export
- fastparquet : for parquet export
other optional dependencies
- PyQt5 : for GUI tool
- pyqtgraph : for GUI tool and Signal plotting
- matplotlib : as fallback for Signal plotting
- cChardet : to detect non-standard unicode encodings
- chardet : to detect non-standard unicode encodings
Benchmarks
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for asammdf-6.2.0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c56417656bbf45e88e0f124d1da8b333808174bbf9aeb19eabf43bd033656de |
|
MD5 | 7a143f3356f3ef2281353ec1ccf4c576 |
|
BLAKE2b-256 | 6393b88e902ab10240d09dd5a1692ea331b953236a5a983584c5a0e662b5c43d |
Hashes for asammdf-6.2.0-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | feb5453825b48a8a029a9e47a3df3e166dd739bc006155a9c1788b6d29ce11e2 |
|
MD5 | 27b50e75d9eab65d68a880a7d387927c |
|
BLAKE2b-256 | d9bfb1ce544ede676eb79661c7ecb14473a4c649bf7e0c6845eddf9ea3c7ac8c |
Hashes for asammdf-6.2.0-cp39-cp39-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 25ab29ee1de2276c2b0e7a83a5f3427e3177bcf62c292f56ae5931c86ae8fdd4 |
|
MD5 | 98ffe8182d990bf8613eb921f7161418 |
|
BLAKE2b-256 | 1a9f7a2269467a673ba8b96aae8504e27ca9e2567af9e4a96a24477657b89d5d |
Hashes for asammdf-6.2.0-cp39-cp39-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12797c19d610d2fac2c7e227096f92ccfdb76fe3cb0e49ff61c3859e8d9d3745 |
|
MD5 | d3c1b2007a654455adc7166df30043f7 |
|
BLAKE2b-256 | b8b3632e0d60949e50c642db5da94899cdb70832e7c62790629084ae36bb7ec9 |
Hashes for asammdf-6.2.0-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32be6fc331058b4f6955608031dc704d729a5eb55142c44062a713671b189e95 |
|
MD5 | 2c4a9530be750dbddc9e10d5d4266885 |
|
BLAKE2b-256 | 795d21b43764b034711062217fe399638009d40cf80c2db1f228955822d04028 |
Hashes for asammdf-6.2.0-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 127f4f9a79d4a2ade7cc13a903a5eb93f8ea784743bea7a6a27ddb571d2a21fb |
|
MD5 | 6f722d1a45b244dea25e8bed94596cb8 |
|
BLAKE2b-256 | df38e5e78a4a6be9a3723c2821ba1c8baae88decc82233fc48efef7a3b2bb040 |
Hashes for asammdf-6.2.0-cp38-cp38-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5d1fe713e02ce6c0770520b89b32169b2eefd63bdb326d97df93d492eb1c9168 |
|
MD5 | cc65096850275056d26789852f31308b |
|
BLAKE2b-256 | 3b02095830ca5d74c4be7479b813244c153e93b214c5f111528917fb620c0f92 |
Hashes for asammdf-6.2.0-cp38-cp38-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5d1dbf1efecb3a1c7b3d7936e617578a868f223fff14f76bb95a4006c4cd9552 |
|
MD5 | 721d9d6ac8749174ae8ce3cf847902f4 |
|
BLAKE2b-256 | 7be1fecf8e129d57be80bfccff83123cb40ac4e6a8dc6c4984a00da48287d76e |
Hashes for asammdf-6.2.0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b88d19187385c3c95c0b3968e4edf8bfdb8eaf6e8d600ef21aba01bc542d158b |
|
MD5 | 860c6f9a2d7e08737394d10f02c19987 |
|
BLAKE2b-256 | 9fbd8342913ad7d16a3d3a03252f1f0351593c8b7f27281a95e9c9718ce4736a |
Hashes for asammdf-6.2.0-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8a451d8a87534ef6897282d7794d1cbe181baff4d4ac56daf904e982ee909436 |
|
MD5 | 8930d67a45f84053537e7c857486db71 |
|
BLAKE2b-256 | 3da7dcd00a4d9dfba59d97e095a04b8e634da6c7b0a4c9ebd054defd6eff7608 |
Hashes for asammdf-6.2.0-cp37-cp37m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2001ac45d13213c0daf652846f6f265d353809e1315ef794502fa22381e7bebe |
|
MD5 | 07dbd8c9ab638b040d2779f2bfa5f66f |
|
BLAKE2b-256 | 419e641f01fa21cbe887ec8b08d5f0037e2ed2d4747c0332e4ff926f4504126e |
Hashes for asammdf-6.2.0-cp37-cp37m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | acc8b8f78efc6286021e9631b4e19c56dbfaf4263c81c7b12789bf59f0da7245 |
|
MD5 | f785131a99e9e6a282ce1ca0b8b22d71 |
|
BLAKE2b-256 | 68765b7d8baf44d25699ef45c7fe5be609ad68331e8f807ca18c6a7e730a11b6 |
Hashes for asammdf-6.2.0-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a1edd2e83ab6c16e3051ac12430b0fe7fd072df0339c87c10941dfca667579bc |
|
MD5 | 5671b744c0b3eca8604eb6393ee5a526 |
|
BLAKE2b-256 | a2879419d8b7d088736d624276ccc3e6643290ab80fc4ebb12d84266f72a6047 |
Hashes for asammdf-6.2.0-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4c9f3b262f6734a2a43e8d749d21e903e6aa597022988ac79d185d56472a44fd |
|
MD5 | 7543008b8ea69e1ccf275c50071556bc |
|
BLAKE2b-256 | c9fb85ee81254c13802442830772930d3bec7b218e9714348d1b5cb4cc9b0fe6 |
Hashes for asammdf-6.2.0-cp36-cp36m-macosx_10_15_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3ba5a1c83589fef7623490b58dd7dba1490e78b38b13e1bed2a52a1a78f21223 |
|
MD5 | 9303cf6e4ea4b155fb9cf530791cca54 |
|
BLAKE2b-256 | 691f5e3cd4c3e7623263066ac55a6ce2c2b09fa5fb78fcbce4a688efcac29c8a |
Hashes for asammdf-6.2.0-cp36-cp36m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 18420f7a72112090f7f5a10d8cbab187d49e7366e3ce6fa1872c8f3bf59a1032 |
|
MD5 | 07e7ebf6249eb97f7e534f27ee5e1c39 |
|
BLAKE2b-256 | 7066fdd91dc6e97fbb3dada43e1bd439ee109af8c7ac5531dc9140db2c449fdb |