Skip to main content

A simple event tracking system.

Project description

Part of edX code.

Event Tracking library build-status

The event-tracking library tracks context-aware semi-structured system events. It captures and stores events with nested data structures in order to truly take advantage of schemaless data storage systems.

Key features:

  • Multiple backends - define custom backends that can be used to persist your event data.

  • Nested contexts - allows data to be injected into events even without having to pass around all of said data to every location where the events are emitted.

  • Django integration - provides a Django app that allows context aware events to easily be captured by multi-threaded web applications.

  • MongoDB integration - support writing events out to a mongo collection.

Example:

from eventtracking import tracker

tracker = tracker.get_tracker()
tracker.enter_context('outer', {'user_id': 10938})
tracker.emit('navigation.request', {'url': 'http://www.edx.org/some/path/1'})

with tracker.context({'user_id': 11111, 'session_id': '2987lkjdyoioey'}):
    tracker.emit('navigation.request', {'url': 'http://www.edx.org/some/path/2'})

tracker.emit(
    'address.create',
    {
        'name': 'foo',
        'address': {
            'postal_code': '90210',
            'country': 'United States'
        }
    }
)

Running the above example produces the following events:

{
    "name": "navigation.request",
    "timestamp": ...,
    "context": {
        "user_id": 10938
    },
    "data": {
        "url": "http://www.edx.org/some/path/1"
    }
},
{
    "name": "navigation.request",
    "timestamp": ...,
    "context": {
        "user_id": 11111,
        "session_id": "2987lkjdyoioey"
    },
    "data": {
        "url": "http://www.edx.org/some/path/2"
    }
},
{
    "name": "address.create",
    "timestamp": ...,
    "context": {
        "user_id": 10938
    },
    "data": {
        "name": "foo",
        "address": {
            "postal_code": "90210",
            "country": "United States"
        }
    }
}

Configuration

Configuration for event-tracking takes the form of a tree of backends. When a Tracker is instantiated, it creates a root RoutingBackend object using the top-level backends and processors that are passed to it. (Or in the case of the DjangoTracker, the backends and processors are constructed according to the appropriate Django settings.)

In this RoutingBackend, each event is first passed through the chain of processors in series, and then distributed to each backend in turn. Theoretically, these backends might be the Mongo, Segment, or logger backends, but in practice these are wrapped by another layer of RoutingBackend. This allows each one to have its own set of processors that are not shared with other backends, allowing independent filtering or event emit cancellation.

Asynchronous Routing

Considering the volume of the events being generated, we would want to avoid processing events in the main thread that could cause delays in response depending upon the operations and event processors.

event-tracking provides a solution for this i.e. AsyncRoutingBackend. It extends RoutingBackend but performs its operations asynchronously.

It can:

  • Process event through the configured processors.

  • If the event is processed successfully, pass it to the configured backends.

Handling the operations asynchronously would avoid overburdening the main thread and pass the intensive processing tasks to celery workers.

Limitations: Although backends for RoutingBackend can be configured at any level of EVENT_TRACKING_BACKENDS configuration tree, AsyncRoutingBackend only supports backends defined at the root level of EVENT_TRACKING_BACKENDS setting. It is also only possible to use it successfully from the default tracker.

An example configuration for AsyncRoutingBackend is provided below:

EVENT_TRACKING_BACKENDS = {
    'caliper': {
        'ENGINE':  'eventtracking.backends.async_routing.AsyncRoutingBackend',
        'OPTIONS': {
            'backend_name': 'caliper',
            'processors': [
                {
                    'ENGINE': 'eventtracking.processors.regex_filter.RegexFilter',
                    'OPTIONS':{
                        'filter_type': 'allowlist',
                        'regular_expressions': [
                            'edx.course.enrollment.activated',
                            'edx.course.enrollment.deactivated',
                        ]
                    }
                }
            ],
            'backends': {
                'caliper': {
                    'ENGINE': 'dummy.backend.engine',
                    'OPTIONS': {
                        ...
                    }
                }
            },
        },
    },
    'tracking_logs': {
        ...
    }
    ...
}

Event Bus Routing

event-tracking provides a solution for routing events to the Event Bus using the EventBusBackend. It extends RoutingBackend but sends events to the Event Bus.

It can:

  • Process event through the configured processors.

  • If the event is allowed via EVENT_BUS_TRACKING_LOGS, send it to the Event Bus.

Make sure to enable the setting: SEND_TRACKING_EVENT_EMITTED_SIGNAL to allow the EventBusBackend to send events to the Event Bus.

An example configuration for EventBusBackend is provided below:

EVENT_TRACKING_BACKENDS = {
    'xapi': {
        'ENGINE':  'eventtracking.backends.event_bus.EventBusBackend',
        'OPTIONS': {
            'backend_name': 'xapi',
            'processors': [
                {
                    'ENGINE': 'eventtracking.processors.regex_filter.RegexFilter',
                    'OPTIONS':{
                        'filter_type': 'allowlist',
                        'regular_expressions': [
                            'edx.course.enrollment.activated',
                            'edx.course.enrollment.deactivated',
                        ]
                    }
                }
            ],
            'backends': {
                'xapi': {
                    'ENGINE': 'dummy.backend.engine',
                    'OPTIONS': {
                        ...
                    }
                }
            },
        },
    },
    'tracking_logs': {
        ...
    }
    ...
}

EVENT_BUS_TRACKING_LOGS = [
    'edx.course.enrollment.activated',
    'edx.course.enrollment.deactivated',
]

Roadmap

In the very near future the following features are planned:

  • Dynamic event documentation and event metadata - allow event emitters to document the event types, and persist this documentation along with the events so that it can be referenced during analysis to provide context about what the event is and when it is emitted.

Documentation

Latest documentation (Hosted on Read the Docs)

License

The code in this repository is licensed under version 3 of the AGPL unless otherwise noted.

Please see LICENSE.txt for details.

How to Contribute

Contributions are very welcome.

Please read How To Contribute for details.

Reporting Security Issues

Please do not report security issues in public. Please email security@openedx.org

Mailing List and IRC Channel

You can discuss this code on the edx-code Google Group or in the edx-code IRC channel on Freenode.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

event_tracking-3.3.0.tar.gz (44.0 kB view details)

Uploaded Source

Built Distribution

event_tracking-3.3.0-py3-none-any.whl (54.8 kB view details)

Uploaded Python 3

File details

Details for the file event_tracking-3.3.0.tar.gz.

File metadata

  • Download URL: event_tracking-3.3.0.tar.gz
  • Upload date:
  • Size: 44.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for event_tracking-3.3.0.tar.gz
Algorithm Hash digest
SHA256 7e9d8f92c7dbd088bb6ddc0c349c1c69845e088cfd874d88925c157e37bd8c41
MD5 32740fb0add1fa332ebe8c6f3bb3b3c3
BLAKE2b-256 2298470a53eb2c7d46973f8947a8201b610704b1ba51f97a2a27a36cce7b03fa

See more details on using hashes here.

File details

Details for the file event_tracking-3.3.0-py3-none-any.whl.

File metadata

  • Download URL: event_tracking-3.3.0-py3-none-any.whl
  • Upload date:
  • Size: 54.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for event_tracking-3.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 e0462381ed1dbb75cec5bf2ad4daef67fcbb0e054b6e05a750403b3498493bcb
MD5 718d67e357f641e587393c225341ee55
BLAKE2b-256 aa1c8419ffb0ca0e1cf684a58ace600b2400af0c5835c43134269ee42f4c7756

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page