openedx-caliper-tracking 0.14.3

Installation

To install openedx-caliper-tracking in your Open edX instance, please add the following line to your requirements file. (For most Open edX installations it should be located at edx-platform/requirements/edx/base.txt).

openedx-caliper-tracking==0.14.3

For manual installation:

pip install openedx-caliper-tracking

Configuration

To enable and use openedx-caliper-tracking:

1. Add ENABLE_EVENT_CALIPERIZATION flag under FEATURES in the following files:

• lms.env.json

• cms.env.json

These files should be located at /edx/app/edxapp/ directory, see the example below:

"FEATURES": {
    ...

    "ENABLE_EVENT_CALIPERIZATION": true,

    ...
}

2. Add the following lines of code:

if FEATURES.get('ENABLE_EVENT_CALIPERIZATION'):
    INSTALLED_APPS.insert(
        INSTALLED_APPS.index('eventtracking.django.apps.EventTrackingConfig'),
        'openedx_caliper_tracking'
    )

in the following files:

• lms/envs/production.py (aws.py for hawthorn release)

• cms/envs/production.py (aws.py for hawthorn release)

Note:

Must make sure that after doing all the required changes in env (lms.env.json and cms.env.json) and auth (lms.auth.json and cms.auth.json) files you restart the server to reflect the changes that you have made.

Sending logs to external APIs (Optional)

There are two ways we can send caliper transformed logs to any external API

• Rest API

• Kafka

"FEATURES": {
    ...

    "ENABLE_CALIPER_EVENTS_DELIVERY": true,

    ...
}

if FEATURES.get('ENABLE_CALIPER_EVENTS_DELIVERY'):
    CALIPER_DELIVERY_ENDPOINT = ENV_TOKENS.get('CALIPER_DELIVERY_ENDPOINT')
    CALIPER_DELIVERY_AUTH_TOKEN = AUTH_TOKENS.get('CALIPER_DELIVERY_AUTH_TOKEN')

"FEATURES": {
    ...

    "ENABLE_KAFKA_FOR_CALIPER": true,

    ...
}

"CALIPER_KAFKA_SETTINGS": {
    "PRODUCER_CONFIG": {
        "bootstrap_servers": [
            "<List of Kafka Brokers URLs>"
        ],
        ...
    },

    "TOPIC_NAME": "<Kafka Topic Name>",

    "ERROR_REPORT_EMAILS": [
        "<Reporting Email Address 1>",
        "<Reporting Email Address 2>",
        ...
    ]
    "MAXIMUM_RETRIES": <An Integer>
},

"CALIPER_KAFKA_AUTH_SETTINGS": {
    "PRODUCER_CONFIG": {
        ...
        "sasl_plain_username": "<Username>",
        "sasl_plain_password": "<Password>",
        "security_protocol": "<Secuirty Protocol>",
        "ssl_cafile": "<Path/to/the/ca/file>",
        ...
    }
}

if FEATURES.get('ENABLE_KAFKA_FOR_CALIPER'):
    CALIPER_KAFKA_SETTINGS = ENV_TOKENS.get('CALIPER_KAFKA_SETTINGS')
    CALIPER_KAFKA_AUTH_SETTINGS = AUTH_TOKENS.get('CALIPER_KAFKA_AUTH_SETTINGS')

Location of Transformed Logs

Note: This doesn’t work locally.

Transformed events are logged using 'logging.handlers.SysLogHandler' with 'facility: local2'.

We need to create output files manually and set appropriate permissions for syslog user. To do so, please follow the steps below:

1. Create a log file with read/write permissions given to syslog user (e.g: /edx/var/log/caliper-analytics/caliper.log).

cd /edx/var/log
mkdir -p caliper-analytics && cd caliper-analytics
touch caliper.log
chown syslog caliper.log

2. Create a mapping for 'local2' in the configuration files present in /etc/rsyslog.d/ (e.g: in 99-edx.conf).

local2.*                 /edx/var/log/caliper-analytics/caliper.log;tracking

3. Run the following command on server to restart the rsyslog daemon:

sudo service rsyslog restart

Location of Logs Whose Delivery to Kafka is failed

Note:

This doesn’t work locally. Do this only if you are sending logs to external source using Kafka broker API.

Transformed events are logged using 'logging.handlers.SysLogHandler' with 'facility: local3'.

We need to create output files manually and set appropriate permissions for syslog user. To do so, please follow the steps below:

1. Create a log file with read/write permissions given to syslog user (e.g: /edx/var/log/caliper-analytics/delivery_failure.log).

cd /edx/var/log
mkdir -p caliper-analytics && cd caliper-analytics
touch delivery_failure.log
chown syslog delivery_failure.log

2. Create a mapping for 'local3' in the configuration files present in /etc/rsyslog.d/ (e.g: in 99-edx.conf).

local3.*                 /edx/var/log/caliper-analytics/delivery_failure.log;tracking

3. Run the following command on server to restart the rsyslog daemon:

sudo service rsyslog restart

Running Tests Locally

To run the unit tests of this app locally, follow the following steps:

• Clone the repository

git clone git@github.com:ucsd-ets/openedx-caliper-tracking.git

• Run the following command in the same directory in which you have cloned the repository

sudo ./openedx-caliper-tracking/openedx_caliper_tracking/tests/local_test_script.sh

License

The code in this repository is licensed under the GPL v3.0 unless otherwise noted. Please see LICENSE for details.

How To Contribute

To contribute, please make a pull request in the repository on Github . If you have any questions or issues, please feel free to open an issue on Github: Open edX Caliper Tracking.

Contributors

• Muhammad Zeeshan

• Tasawer Nawaz

• Aroosha Arif

• Osama Arshad

• Danial Malik

• Hamza Farooq

• Hassan Tariq

• Muhammad Umar Khan

• Tehreem Sadat

• Muhammad Arslan

• Saad Ali

• Husnain Raza Ghaffar