Skip to main content

Modular DAQ system

Project description


A modular, parallel-ready data acquisition framework

Xylem systems consist of a collection of software components connected to represent the flow of information (data and commands) through the experiment. Each component has a particular relation to data (e.g. Producer, Consumer, Aggregator) which determines the available connections and functionality. For example, Producers introduce new data into the xylem system, and Consumers remove data from the system (e.g. into offline storage). The components work together to get data through the pipeline as smoothly as possible.

Xylem components are designed to fit into code that you already have which controls your experimental equipment and processes your data. For example, if you have code which interfaces with an instrument and prints the data to the console, you can replace the print statement with a call to Producer.produce(). Then, the xylem Producer will send off the data to the downstream xylem components while your code continues running.

The xylem Core is a special component which monitors and commmunicates with all of the other components in the system. It determines if a component has crashed or otherwise become unresponsive, and it also can send state updates and arbitrary messages to the components.

Human operators can access the functionality of the Core component using the Controller component. This component does not participate in the data flow; rather, it allows the human operator to send commands to the system (via the Core) and query its state (also via the Core). You can design your system to respond to any set of commands you'd like, such as "Begin run," "Initialize instrumentation," "Perform calibration," or anything else that makes sense in your system. Internally, your code would translate your commands into state updates that will be sent to the Core. You provide the logic; xylem simply makes sure the messages reach their intended destination.

DAQ Components

Fundamental building blocks for basic setups:

  • Data producer: produces data. Example: digitizers

  • Data consumer: accepts data inputs and sends no output to other xylem components. Examples: offline data storage, live data display

  • Data aggregator: collects data from multiple data producers and supplies the data to multiple data consumers

Compound building blocks for more advanced setups:

  • Data producer group: a collection of data producers whose output is aggregated before being sent to the data aggregator, so that the collection can be treated as a single data producer. Example: an array of identical sensors

Example basic architecture

  • Data producer
  • Data aggregator
  • Data consumer

Example complex architecture

  • Data producer group 1*
    • Data producer
    • Data producer
    • ...
  • Data producer group 2*
    • Data producer
    • Data producer
    • ...
  • Data aggregator 1
    • Data consumer 1+
    • Data consumer 2
    • ...
  • Data aggregator 2
    • Data consumer 1+
    • Data consumer 3
    • ...

*: The data producer groups are located after the data producers in the data flow.

+: A single data consumer instance can receive data from multiple data aggregators.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

xylem-daq-0.3.2.tar.gz (15.1 kB view hashes)

Uploaded source

Built Distributions

xylem_daq-0.3.2-py3-none-any.whl (19.4 kB view hashes)

Uploaded py3

xylem_daq-0.3.2-py2-none-any.whl (19.4 kB view hashes)

Uploaded py2

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page