Skip to main content

Library for opening Nalu Scientific data files

Project description

NaluACQ

naluacq is a Rust crate for I/O with Nalu Scientific acquisitions in the GbE-compatible format.

The crate encompasses the following:

  • Reading acquisitions
  • Writing acquisitions (mainly used by naludaq_rs)
  • Parsing events from acquisitions
  • Calibration of events
  • Exporting acquisitions to other formats

Some of these features are made available in the Python (PyO3) bindings for this crate.

Structure of Acquisitions

An acquisition in the new format is a directory containing multiple files describing various aspects of the acquisition.

  • {n}.bin - binary event data file "n" (up to 500 MB) containing raw events back-to-back
  • {n}.idx - index file containing the offset and length of each event in the corresponding .bin file
  • metadata.yml - user-defined string metadata. NaluDAQ uses this to store board parameters and registers.
  • readout_metadata - more user-defined binary metadata. NaluDAQ uses this for extra information about the readout.
  • [pedestals_calibration] - optional binary file containing pedestal calibration data
  • [timing_calibration] - optional binary file containing timing calibration data
  • [adc2mv_calibration] - optional binary file containing ADC to mV calibration data

Event Storage

Events are stored in groups of 500 MB maximum known as "chunks." Each chunk is represented using two files: a .bin file which contains the raw event data and an .idx file which points to each event in the .bin file.

Bin File

Events are stored back-to-back in the .bin files as unparsed binary data. A maximum of 500 MB (at the time of writing) is allowed to be stored in a single .bin file. The rough structure is as follows:

  1. Metadata Sector
    • 8-byte header indicating file version type and sector length
    • user-defined metadata stored as raw bytes
  2. Data Sector
    • Event 1 (raw bytes)
    • Event 2 (raw bytes)
    • ...

Idx File

Because searching through the entire file for a specific event would be painfully slow, an index file is used to store the offset and length of each event in the corresponding .bin file. The index file is comprised of several 8-byte entries, each indicating the offset and length of a single event in the .bin file. A specific event can be found by reading the entry at 8 * index bytes and then reading length bytes from the .bin file starting at offset bytes.

Examples

Reading an Acquisition

use naluacq::Acquisition;

let acq = Acquisition::open("path/to/acquisition.acq").unwrap();

let idx = 0;
let raw_event = acq.get(idx).unwrap()

Parsing an Acquisitions

use naluacq::{Acquisition, Aardvarcv3Event, ParseInto};

let acq = Acquisition::open("path/to/acquisition.acq").unwrap();

let idx = 0;
let raw_event = acq.get(idx).unwrap()

let parsed_event: Aardvarcv3Event = raw_event.parse_into().unwrap();

Building the Documentation

cargo doc --open

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

naluacq-0.4.5-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.8 kB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

naluacq-0.4.5-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.7 kB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

naluacq-0.4.5-cp312-none-win_amd64.whl (496.8 kB view details)

Uploaded CPython 3.12 Windows x86-64

naluacq-0.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.0 kB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

naluacq-0.4.5-cp311-none-win_amd64.whl (496.8 kB view details)

Uploaded CPython 3.11 Windows x86-64

naluacq-0.4.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.4 kB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

naluacq-0.4.5-cp310-none-win_amd64.whl (496.8 kB view details)

Uploaded CPython 3.10 Windows x86-64

naluacq-0.4.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.4 kB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

naluacq-0.4.5-cp39-none-win_amd64.whl (497.1 kB view details)

Uploaded CPython 3.9 Windows x86-64

naluacq-0.4.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.2 kB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

naluacq-0.4.5-cp38-none-win_amd64.whl (497.7 kB view details)

Uploaded CPython 3.8 Windows x86-64

naluacq-0.4.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.6 kB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

naluacq-0.4.5-cp37-none-win_amd64.whl (497.8 kB view details)

Uploaded CPython 3.7 Windows x86-64

naluacq-0.4.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (668.9 kB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

File details

Details for the file naluacq-0.4.5-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 99c873be79b973af04d1fcf9b81283b8e3dec403e7ecac34a39ba44327803313
MD5 5c55ef5b3eb9acbe091f1edcf9c5cf97
BLAKE2b-256 98e79bdcb562379c48b73fe5507330bdd04c84118d078220d950d718512eb68d

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a6ae7d327543e444cce75e7ec9ec7303f00c11cad5c606b85630e571ecf1e458
MD5 c61e1eba0348bb0964c67288aac1dfc9
BLAKE2b-256 66fa7162727736f74f70f400250bdfe91b61484247f689d57703256a6cb2d075

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 cf98f792e932ee41c2af8a2be8dabed16c43c1ed08af7df1518270ae71147f05
MD5 91c427720e311279f2479f90e320adf2
BLAKE2b-256 ba88a7b1406617a9410b7ec4e9bcd14328d1acdf1222e383bbc0db775ec3143a

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d061214465b8186993407dc5b3a38559c0260b37ce0b8b560e9423db9ee5ccca
MD5 e9e0ec71a1485ae18da7daddd94c343e
BLAKE2b-256 f860985f27083f5b27c6048db89675c3b1f9dcd89b66ff0b1d0fd3adb4913412

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 d006130701603ed21ad379e75a8b730f57ce5db9b1b6148a1caca60e44f6470b
MD5 fb20993c99e95bef1d115409f95f1a42
BLAKE2b-256 1bf3bfa4d392d693d84783e006d4baa18805547da4ff83bf9d4bb8a57520487b

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 52b1ea0bdac5d3d3519f0acd1f5656c5bcc43ce3e59d48164424853f57db3bca
MD5 0fa90d92054e60354bf1cb8bb4d071c0
BLAKE2b-256 65baeabbd3b1eb661971ceff526c1496c75494f1db8180c77be3879906fbcbc6

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 62940dc9a9d4c6331b40f8fcb0ec23484a0472cdbeca62b93bffc78f7e67c1c8
MD5 7a4609fc818d107e4bd6195f2d223423
BLAKE2b-256 6682cf96a9cf74f10078b17e63ccb3ddb8fc783194a296b5e1ebaecdf0d8299b

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 81c2cfe27d0d23c9f9c7386c2ce3d80eb23f38e683d746e84bbfd33b653676a3
MD5 38764974e16d7bbba845f32bee97043c
BLAKE2b-256 75914da126517b363167dc8132a342fa0040806ebedabb3ddccdb0d457f3c425

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 5f4f2e72d5a40fb000a3565dd55b0f05bb663db965060210f3c40ba420be8487
MD5 ab9b0063fbb1e4a0e55701cbdd85d92b
BLAKE2b-256 1fd8f77ef972a68ff332a2cd01a2678365b0354c2e6eb203ca47a0dd0df2ee57

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 37663fcfaa056e0bc9fe992d00adb4ca11a140c8a405c6f07d6969846cc14d98
MD5 904eadd9bca2e088a6892ee4d419569a
BLAKE2b-256 653f4a777e385df8de062dc8635f3fc4be76cec4ca63695523508cc32fd32177

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 af39951ccc2da6980c626bfbd18c37a9bcd18b5cf1058e899a393cee94f102d7
MD5 97fe15db664a2f91f25726ae439c88cf
BLAKE2b-256 538d9e43beaedfaedff85c905a71ca10362801355c2a08dfdeb308f906945aea

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ab16fec2c30f542bef6e7352f7e4e0651aa91f3db534536e858b62b1600f93e0
MD5 58b40e0d357041823b4eb3f614f50cd7
BLAKE2b-256 527b95921c5b756eba665705f0ab3c34f90b500ad47bc0d72ed97ced3c1d4a0e

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 fd5d1f80b2ad76de5771cd467a1a6c7add81a7cc97b15523c5f212c83b301a63
MD5 ff55558fca494651e6069b7224029a2d
BLAKE2b-256 6c8917d6ea79c825595a4040b847435fba254ab92eb072a06ce97a5476b8e17e

See more details on using hashes here.

File details

Details for the file naluacq-0.4.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for naluacq-0.4.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 41db7c47abe014f32589e10c34715b25983586b8fc4daee0e28019947b5a7b8b
MD5 ae4a2c9119861fa797e633d9aa4a4668
BLAKE2b-256 42cb45c0c4429917ebd0ce114aa8352d3d1f25585b876fcad0df23b59514e392

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page