Skip to main content

A highly-scalable, distributed metric data processing framework based on RabbitMQ

Project description

BSD 3-clause Python package Code style: black PyPI PyPI - Wheel Docker pulls Documentation

metricq - python libraries

This is a python implementation of the MetricQ protocol. It allows you to write Sources and Sinks to easily send and receive data over the MetricQ infrastructure.

Installation

Install the package from PyPI:

$ pip install metricq

Examples

The examples directory contains some basic examples. To play around with them, check out a copy of this repository and (in your favourite venv) install their dependencies:

$ pip install -e '.[examples]'

A simple Source is implemented in metricq_source.py, as is a Sink in metricq_sink.py. We will use the former to produce data for a metric called test.py.dummy, which we will then receive and print with the latter.

Assuming a MetricQ instance is reachable at localhost, configure a client(consult the documentation of your favourite config provider on how to do that) named source-py-dummy to produce values with a frequency of 0.5Hz (i.e. every 2 seconds) :

{
    "rate": 0.5
}

To start the Source, run:

$ ./examples/metricq_source.py --server 'amqp://localhost/' --token 'source-py-dummy'

This should now send values for the metric test.py.dummy in 2-second intervals. To see (in detail) what's going on, add -v DEBUG to the arguments above.

On the other side, run

$ ./examples/metricq_sink.py --server 'amqp://localhost/' --metrics 'test.py.dummy'

and you should see new values for the metric test.py.dummy appear every 2 seconds.

Tools and utility scripts

The repository metricq/metricq-tools contains a collection of tools and utility scripts to monitor and administrate a MetricQ network. Install them from PyPI:

$ pip install metricq-tools

Development setup

Clone the repository, and in a virtual environment run

$ pip install -e '.[dev]'

This will install all tools necessary for testing and linting. To test code manually, run pytest. Format code using black and isort, or lint with flake8. To make sure a source distribution (sdist) contains the correct files, run check-manifest. Tools are configured in setup.cfg respectively pyproject.toml.

To test code in a fresh environment, run the tox test harness:

$ tox

This runs the same step as our CI does. If tox passes locally there's high chances that CI steps will pass too.

We recommend to install our pre-commit hooks:

$ pre-commit install

This way commits that fail tests or do not comply with our code style are rejected right away.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

metricq-5.4.0-py3-none-any.whl (80.0 kB view details)

Uploaded Python 3

File details

Details for the file metricq-5.4.0-py3-none-any.whl.

File metadata

  • Download URL: metricq-5.4.0-py3-none-any.whl
  • Upload date:
  • Size: 80.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.9.21

File hashes

Hashes for metricq-5.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ebda54cc7f1c15c5d0f3b57329bc7461ae79aa8606f49cacb7165c9f4743fe9f
MD5 f7e5049547575d447e3ed5565f671932
BLAKE2b-256 8e8231a4e485802bd877bae952645d07636e38ed13db241447aad297783a70cf

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page