Skip to main content

A pytest plugin for instrumenting test runs via OpenTelemetry

Project description

pytest-opentelemetry

Instruments your pytest runs, exporting the spans and timing via OpenTelemetry.

Why instrument my test suite?

As projects grow larger, perhaps with many contributors, test suite runtime can be a significant limiting factor to how fast you and your team can deliver changes. By measuring your test suite's runtime in detail, and keeping a history of this runtime in a visualization tool like Jaeger, you can spot test bottlenecks that might be slowing your entire suite down.

Additionally, pytest makes an excellent driver for integration tests that operate on fully deployed systems, like your testing/staging environment. By using pytest-opentelemetry and configuring the appropriate propagators, you can connect traces from your integration test suite to your running system to analyze failures more quickly.

Even if you only enable pytest-opentelemetry locally for occasional debugging, it can help you understand exactly what is slowing your test suite down. Did you forget to mock that requests call? Didn't realize the test suite was creating 10,000 example accounts? Should that database setup fixture be marked scope=module? These are the kinds of questions pytest-opentelemetry can help you answer.

pytest-opentelemetry works even better when testing applications and libraries that are themselves instrumented with OpenTelemetry. This will give you deeper visibility into the layers of your stack, like database queries and network requests.

Installation and usage

pip install pytest-opentelemetry

Installing a library that exposes a specific pytest-related entry point is automatically loaded as a pytest plugin. Simply installing the plugin should be enough to register it for pytest.

Using the --export-traces flag enables trace exporting (otherwise, the created spans will only be tracked in memory):

pytest --export-traces

By default, this exports traces to http://localhost:4317, which will work well if you're running a local OpenTelemetry Collector exposing the OTLP gRPC interface. You can use any of the OpenTelemetry environment variables to adjust the tracing export or behavior:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://another.collector:4317
pytest --export-traces

Only the OTLP over gRPC exporter is currently supported.

pytest-opentelemetry will use the name of the project's directory as the OpenTelemetry service.name, but it will also respect the standard OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES environment variables. If you would like to permanently specify those for your project, consider using the very helpful pytest-env package to set these for all test runs, for example, in your pyproject.toml:

[tool.pytest.ini_options]
env = [
    "OTEL_RESOURCE_ATTRIBUTES=service.name=my-project",
]

If you are using the delightful pytest-xdist package to spread your tests out over multiple processes or hosts, pytest-opentelemetry will automatically unite them all under one trace. If this pytest run is part of a larger trace, you can provide a --trace-parent argument to nest this run under that parent:

pytest ... --trace-parent 00-1234567890abcdef1234567890abcdef-fedcba0987654321-01

Visualizing test traces

One quick way to visualize test traces would be to use an OpenTelemetry Collector feeding traces to Jaeger. This can be configured with a minimal Docker Compose file like:

version: "3.8"
services:
  jaeger:
    image: jaegertracing/all-in-one:1.33
    ports:
    - 16686:16686    # frontend
    - 14250:14250    # model.proto
  collector:
    image: otel/opentelemetry-collector-contrib:0.49.0
    depends_on:
    - jaeger
    ports:
    - 4317:4317      # OTLP (gRPC)
    volumes:
    - ./otelcol-config.yaml:/etc/otelcol-contrib/config.yaml:ro

With this otelcol-config.yaml:

receivers:
  otlp:
    protocols:
      grpc:

processors:
  batch:

exporters:
  jaeger:
    endpoint: jaeger:14250
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger]

Developing

Two references I keep returning to is the pytest guide on writing plugins, and the pytest API reference:

These are extremely helpful in understanding the lifecycle of a pytest run.

To get setup for development, you will likely want to use a "virtual environment", using great tools like virtualenv or pyenv.

Once you have a virtual environment, install this package for editing, along with its development dependencies, with this command:

pip install -e '.[dev]'

When sending pull requests, don't forget to bump the version in setup.cfg.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest-opentelemetry-1.0.0.tar.gz (17.2 kB view details)

Uploaded Source

Built Distribution

pytest_opentelemetry-1.0.0-py3-none-any.whl (9.9 kB view details)

Uploaded Python 3

File details

Details for the file pytest-opentelemetry-1.0.0.tar.gz.

File metadata

  • Download URL: pytest-opentelemetry-1.0.0.tar.gz
  • Upload date:
  • Size: 17.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for pytest-opentelemetry-1.0.0.tar.gz
Algorithm Hash digest
SHA256 fbb0fe777998e510a6556c17c12a77f474f36e21774f81a0ab4333fa12f4e7df
MD5 32fe201ea5471d8ddb60ba8eed8d6601
BLAKE2b-256 3c59dc7f803384a320ef8b8e2196444dc60945a3a629eb67da430583d40b4ded

See more details on using hashes here.

File details

Details for the file pytest_opentelemetry-1.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for pytest_opentelemetry-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 cd649d235500aa8237871aca5d9444293576680b9d3b3c65341208f3b76a2f59
MD5 cd38b22f999e9a227d34eecb74308d0e
BLAKE2b-256 9b8e874c56017e61b3f863c12f653d98a6ffec5140e5d3ebc4b8499e65610352

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page