Google Analytics Driver

• Documentation: https://metrics.operas-eu.org/docs/google-analytics

This driver allows programmatic retrieval of Google Analytics stats reports in order to generate normalised publication-level usage reports.

The driver is made of two modules: the first one obtains statistics from GA's API and stores them in a directory (CACHEDIR); the second reads from cache, normalises the reports, and outputs to a different directory (OUTDIR). We recommend running this driver in a docker container and mapping both CACHEDIR and OUTDIR to persistent volumes.

Setup

Requirements

Identifier normalisation is performed using an instance of hirmeos/identifier_translation_service - you must first setup this API.

API Service Account Key

Obtain the private key of a service account linked to your google analytics views: https://developers.google.com/analytics/devguides/reporting/core/v4/quickstart/service-py

The private key will be inside a json file. We must provide the path to this file using the env variable KEY_PATH (see "Environment variables" below). If you are running the driver using docker you will need to map the local json file to the KEY_PATH location in the container (see "Run via crontab" below).

Environment variables

The following environment variables must be set. You can find a template in ./config/config.env.example.

Example config.env file

MODES=["measure":"https://metrics.operas-eu.org/obp-pdf/sessions/v1","name":"obp-htmlreader","prefix":"https://www.openbookpublishers.com/htmlreader","startDate":"2014-02-26","config":[{"name":"view-id","value":"012345678"},{"name":"metric","value":"ga:uniquePageViews"},{"name":"dimension","value":"ga:pagePathLevel2"},{"name":"dimension","value":"ga:countryIsoCode"},{"name":"filter","value":"^/htmlreader/"}],"regex":["https:\\/\\/www\\.openbookpublishers\\.com\\/htmlreader\\/(?:[0-9]{3}-)?[0-9]{1,5}-[0-9]{1,7}-[0-9]{1,6}-[0-9]"]}]
EXCLUDED_URLS=[]
OUTDIR=/usr/src/app/output
CACHEDIR=/usr/src/app/cache
KEY_PATH=/usr/src/app/config/key.json
URI_API_ENDP=https://identifier.translation.service/translate
AUTH_API_ENDP=https://authentication.service/tokens
URI_API_USER=admin_user@openbookpublishers.com
URI_API_PASS=some_secret_password
URI_SCHEME=info:doi
COUNTRY_URI_SCHEME=urn:iso:std:3166:-2
URI_STRICT=false
CUTOFF_DAYS=1

The MODES env variable

You must define a JSON array inMODES, with at least one record. The driver will iterate through the array, performing its task once per mode; in a typical case there will only be one entry in the array, however this configuration allows one single driver to query reports from multiple google analytics accounts.

Each entry of the MODES array must contain values for measure, name, startDate, and config.

The config attribute

Example:

MODES=[{"measure":"https://metrics.operas-eu.org/obp-pdf/sessions/v1","name":"obp-htmlreader","prefix":"https://www.openbookpublishers.com/htmlreader","startDate":"2015-01-01","config":[{"name":"view-id","value":"012345678"},{"name":"metric","value":"ga:uniquePageViews"},{"name":"dimension","value":"ga:pagePathLevel2"},{"name":"dimension","value":"ga:countryIsoCode"},{"name":"filter","value":"^/htmlreader/"}],"regex":["https:\\/\\/www\\.openbookpublishers\\.com\\/htmlreader\\/(?:[0-9]{3}-)?[0-9]{1,5}-[0-9]{1,7}-[0-9]{1,6}-[0-9]"]}]

Run via crontab

0 0 * * 0 docker run --rm --name "google_analytics_driver" --env-file /path/to/config.env -v google_analytics_cache:/usr/src/app/cache -v metrics:/usr/src/app/output openbookpublishers/google_analytics_driver:2

• --rm is used to delete the container once it exists;
• --name is completely optional (it will get receive a random name otherwise);
• --env-file is the path to the config file (in the local machine);
• -v is to add a volume (to persist data). We have three of these: google_analytics_cache will store the results of the API queries to GA; metrics stores the output of the driver (the normalised CSV files); and the last one (key.json) is a mapping of the local json file containing the API credentials, to the location in the container specified in KEY_PATH.