Skip to main content

ASAM MDF measurement data file parser

Project description

asammdf is a fast parser and editor for ASAM (Association for Standardization of Automation and Measuring Systems) MDF (Measurement Data Format) files.

asammdf supports MDF versions 2 (.dat), 3 (.mdf) and 4 (.mf4).

asammdf works on Python >= 3.7 (for Python 2.7, 3.4 and 3.5 see the 4.x.y releases)

Status

Continuous Integration Coveralls Codacy ReadTheDocs
continuous integration Coverage Status Codacy Badge Documentation Status
PyPI conda-forge
PyPI version conda-forge version

Project goals

The main goals for this library are:

  • to be faster than the other Python based mdf libraries
  • to have clean and easy to understand code base
  • to have minimal 3-rd party dependencies

Features

  • create new mdf files from scratch

  • append new channels

  • read unsorted MDF v3 and v4 files

  • read CAN and LIN bus logging files

  • extract CAN and LIN signals from anonymous bus logging measurements

  • filter a subset of channels from original mdf file

  • cut measurement to specified time interval

  • convert to different mdf version

  • export to HDF5, Matlab (v4, v5 and v7.3), CSV and parquet

  • merge multiple files sharing the same internal structure

  • read and save mdf version 4.10 files containing zipped data blocks

  • space optimizations for saved files (no duplicated blocks)

  • split large data blocks (configurable size) for mdf version 4

  • full support (read, append, save) for the following map types (multidimensional array channels):

    • mdf version 3 channels with CDBLOCK

    • mdf version 4 structure channel composition

    • mdf version 4 channel arrays with CNTemplate storage and one of the array types:

      • 0 - array
      • 1 - scaling axis
      • 2 - look-up
  • add and extract attachments for mdf version 4

  • handle large files (for example merging two fileas, each with 14000 channels and 5GB size, on a RaspberryPi)

  • extract channel data, master channel and extra channel information as Signal objects for unified operations with v3 and v4 files

  • time domain operation using the Signal class

    • Pandas data frames are good if all the channels have the same time based
    • a measurement will usually have channels from different sources at different rates
    • the Signal class facilitates operations with such channels
  • graphical interface to visualize channels and perform operations with the files

Major features not implemented (yet)

  • for version 3

    • functionality related to sample reduction block: the samples reduction blocks are simply ignored
  • for version 4

    • experimental support for MDF v4.20 column oriented storage
    • functionality related to sample reduction block: the samples reduction blocks are simply ignored
    • handling of channel hierarchy: channel hierarchy is ignored
    • full handling of bus logging measurements: currently only CAN and LIN bus logging are implemented with the ability to get signals defined in the attached CAN/LIN database (.arxml or .dbc). Signals can also be extracted from an anonymous bus logging measurement by providing a CAN or LIN database (.dbc or .arxml)
    • handling of unfinished measurements (mdf 4): finalization is attempted when the file is loaded, however the not all the finalization steps are supported
    • full support for remaining mdf 4 channel arrays types
    • xml schema for MDBLOCK: most metadata stored in the comment blocks will not be available
    • full handling of event blocks: events are transferred to the new files (in case of calling methods that return new MDF objects) but no new events can be created
    • channels with default X axis: the default X axis is ignored and the channel group's master channel is used
    • attachment encryption/decryption using user provided encryption/decryption functions; this is not part of the MDF v4 spec and is only supported by this library

Usage

from asammdf import MDF

mdf = MDF('sample.mdf')
speed = mdf.get('WheelSpeed')
speed.plot()

important_signals = ['WheelSpeed', 'VehicleSpeed', 'VehicleAcceleration']
# get short measurement with a subset of channels from 10s to 12s
short = mdf.filter(important_signals).cut(start=10, stop=12)

# convert to version 4.10 and save to disk
short.convert('4.10').save('important signals.mf4')

# plot some channels from a huge file
efficient = MDF('huge.mf4')
for signal in efficient.select(['Sensor1', 'Voltage3']):
   signal.plot()

Check the examples folder for extended usage demo, or the documentation http://asammdf.readthedocs.io/en/master/examples.html

https://canlogger.csselectronics.com/canedge-getting-started/log-file-tools/asammdf-api/

Documentation

http://asammdf.readthedocs.io/en/master

And a nicely written tutorial on the CSS Electronics site

Contributing & Support

Please have a look over the contributing guidelines

If you enjoy this library please consider making a donation to the numpy project or to danielhrisca using liberapay <a href="https://liberapay.com/danielhrisca/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a>

Contributors

Thanks to all who contributed with commits to asammdf:

Installation

asammdf is available on

pip install asammdf
# for the GUI 
pip install asammdf[gui]
# or for anaconda
conda install -c conda-forge asammdf

In case a wheel is not present for you OS/Python versions and you lack the proper compiler setup to compile the c-extension code, then you can simply copy-paste the package code to your site-packages. In this way the python fallback code will be used instead of the compiled c-extension code.

Dependencies

asammdf uses the following libraries

  • numpy : the heart that makes all tick
  • numexpr : for algebraic and rational channel conversions
  • wheel : for installation in virtual environments
  • pandas : for DataFrame export
  • canmatrix : to handle CAN/LIN bus logging measurements
  • natsort
  • lxml : for canmatrix arxml support
  • lz4 : to speed up the disk IO performance

optional dependencies needed for exports

  • h5py : for HDF5 export
  • scipy : for Matlab v4 and v5 .mat export
  • hdf5storage : for Matlab v7.3 .mat export
  • fastparquet : for parquet export

other optional dependencies

  • PyQt5 : for GUI tool
  • pyqtgraph : for GUI tool and Signal plotting
  • matplotlib : as fallback for Signal plotting
  • cChardet : to detect non-standard Unicode encodings
  • chardet : to detect non-standard Unicode encodings
  • pyqtlet : for GPS window
  • isal : for faster zlib compression/decompression

Benchmarks

http://asammdf.readthedocs.io/en/master/benchmarks.html

Project details


Release history Release notifications | RSS feed

This version

7.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

asammdf-7.0.1.tar.gz (614.6 kB view details)

Uploaded Source

Built Distributions

asammdf-7.0.1-cp310-cp310-macosx_10_14_x86_64.whl (679.7 kB view details)

Uploaded CPython 3.10 macOS 10.14+ x86-64

asammdf-7.0.1-cp39-cp39-win_amd64.whl (686.5 kB view details)

Uploaded CPython 3.9 Windows x86-64

asammdf-7.0.1-cp39-cp39-macosx_10_14_x86_64.whl (679.7 kB view details)

Uploaded CPython 3.9 macOS 10.14+ x86-64

asammdf-7.0.1-cp38-cp38-win_amd64.whl (686.5 kB view details)

Uploaded CPython 3.8 Windows x86-64

asammdf-7.0.1-cp38-cp38-macosx_10_14_x86_64.whl (679.7 kB view details)

Uploaded CPython 3.8 macOS 10.14+ x86-64

asammdf-7.0.1-cp37-cp37m-win_amd64.whl (686.5 kB view details)

Uploaded CPython 3.7m Windows x86-64

asammdf-7.0.1-cp37-cp37m-macosx_10_14_x86_64.whl (679.7 kB view details)

Uploaded CPython 3.7m macOS 10.14+ x86-64

File details

Details for the file asammdf-7.0.1.tar.gz.

File metadata

  • Download URL: asammdf-7.0.1.tar.gz
  • Upload date:
  • Size: 614.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1.tar.gz
Algorithm Hash digest
SHA256 4759d558acca3c36ac2f791241d8935de4382ecd4bd44b4fe452871a59b44557
MD5 d21890cdeb824880ae9c41c981ed4ff9
BLAKE2b-256 8dc5b36aaec3b3dbe48a438791eb7d1c1f67f94fe70d73e629cb853e90b54726

See more details on using hashes here.

File details

Details for the file asammdf-7.0.1-cp310-cp310-macosx_10_14_x86_64.whl.

File metadata

  • Download URL: asammdf-7.0.1-cp310-cp310-macosx_10_14_x86_64.whl
  • Upload date:
  • Size: 679.7 kB
  • Tags: CPython 3.10, macOS 10.14+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1-cp310-cp310-macosx_10_14_x86_64.whl
Algorithm Hash digest
SHA256 12e5c449f519ff22e0d4e0bfdb7ae7c6c13c08d1686dd3d126e29dac4721b651
MD5 1d532a31f863c89941d5b2d900b609d7
BLAKE2b-256 f56147414e1bd656d2bb3ab2a8996ce80988a020cd72bca4a7ff00e6e4898dfb

See more details on using hashes here.

File details

Details for the file asammdf-7.0.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: asammdf-7.0.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 686.5 kB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 1fd80d0beaa03739dde469872e817422f74f4564502548d0b10e73527899f380
MD5 7042c23e865d5dfc8137a4c27d7acd5e
BLAKE2b-256 ed65c82c5bcf1c1cfe8a9525406bceda62515ed16f9788d4e0b701990363b49f

See more details on using hashes here.

File details

Details for the file asammdf-7.0.1-cp39-cp39-macosx_10_14_x86_64.whl.

File metadata

  • Download URL: asammdf-7.0.1-cp39-cp39-macosx_10_14_x86_64.whl
  • Upload date:
  • Size: 679.7 kB
  • Tags: CPython 3.9, macOS 10.14+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1-cp39-cp39-macosx_10_14_x86_64.whl
Algorithm Hash digest
SHA256 af66fef07d82f31a4eef3bf9e0726c6e0567be41a544e7030215e12f02abcffe
MD5 bc9476ec0cb379b7ed084df3b50d4194
BLAKE2b-256 df6ef953b45f290ec337528a2ee80e60801948146e8dfa0b370532ce960ae702

See more details on using hashes here.

File details

Details for the file asammdf-7.0.1-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: asammdf-7.0.1-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 686.5 kB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 543200a97cd002bcd433a3ab4cd163f2b046963e8df978b992b8af6afe5ebc55
MD5 f359b4f97a1e463dd6a73978a33144a7
BLAKE2b-256 28d148aff21afc5277d06c2730ced47855ea2fbfc13b63c14f8b24063a901005

See more details on using hashes here.

File details

Details for the file asammdf-7.0.1-cp38-cp38-macosx_10_14_x86_64.whl.

File metadata

  • Download URL: asammdf-7.0.1-cp38-cp38-macosx_10_14_x86_64.whl
  • Upload date:
  • Size: 679.7 kB
  • Tags: CPython 3.8, macOS 10.14+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1-cp38-cp38-macosx_10_14_x86_64.whl
Algorithm Hash digest
SHA256 3421332c06aad816ed8dc24fb5d9584e3fce51571d87e0b6827309b18d9cfb6d
MD5 14872a931a758ce4d8ebd2724c0ee0f0
BLAKE2b-256 dca825161eb8ff4bd871f4aaef5c5ae61923eac2f42c8928af8a509bcdbe317d

See more details on using hashes here.

File details

Details for the file asammdf-7.0.1-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: asammdf-7.0.1-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 686.5 kB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 9afde69bab04561f2eecd7a75875c0e97c52b31d7bec2591fe8a963a338e81e4
MD5 8aedb4c307294608b68853c6d50eb543
BLAKE2b-256 e00c5200f537f4448de8c494d2922f1c17630e7fa5733cade2d8da236a21a5a1

See more details on using hashes here.

File details

Details for the file asammdf-7.0.1-cp37-cp37m-macosx_10_14_x86_64.whl.

File metadata

  • Download URL: asammdf-7.0.1-cp37-cp37m-macosx_10_14_x86_64.whl
  • Upload date:
  • Size: 679.7 kB
  • Tags: CPython 3.7m, macOS 10.14+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for asammdf-7.0.1-cp37-cp37m-macosx_10_14_x86_64.whl
Algorithm Hash digest
SHA256 e8ccde6e7146fd776c28a961a6d6a3d190c114b4396f57eef89ae8162f3fa386
MD5 0c47450beb1aee01fc868e9ada9f4fa4
BLAKE2b-256 246dc71923bde0b08bbc639d8d010a0a66f80a57166c9922851a69d32dce93c8

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page