Skip to main content

Utility for building templated metric extraction queries that can be traversed through time.

Project description

Metric Builder

Utility for building templated metric extraction queries that can be traversed through time.

Prerequisites

You will need the following to run this code:

  • Python 3

Installation

To be determined...

Usage

In order to extract a given metric, a Metric object needs to be instantiated:

metric = Metric(
    query="""
        SELECT count(*) AS total
        FROM `project.dataset.table`
        WHERE DATETIME_TRUNC(created_datetime, DAY) = '{{ reference_time | format_date('%Y-%m-%d') }}'
    """,
    reader = BigQueryReader(json_credentials_path='/path/to/creds.json')
)

The query parameter is a templated query where you can format the reference_time datetime object to the required format using template filters.

The reader parameter is the object that is actually going to connect to the desired database and perform the queries.

The metric object can now be used to fetch metrics for a given point in time as follows:

result = metric.fetch(reference_time=datetime.date(2019, 10, 21))

The result is returned as a list of dictionaries.

Template filters

Jinja2 is used as the templating engine. All built in Jinja filters are thus available. Relevant custom template filters have been added though for convenience:

format_date

Specify format of datetime:

'{{ reference_time | format_date('%Y-%m-%d') }}'

day_delta

Change a given datetime object by a specified number of days:

'{{ reference_time | day_delta(-7) | format_date('%Y-%m-%d') }}'

Readers

Any reader will implement the following method that is used to execute queries:

def execute(self, query) -> List[Dict[str, Any]]:
    ...

BigQueryReader

The underlying client is required to be authenticated with the necessary priviledges to read from the requested BigQuery tables.

If you authenticate with:

gcloud auth login

or

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json"

then you can just instantiate your Reader like this:

reader = BigQueryReader()

The other option is to explicitly authenticate with a service account key file:

reader = BigQueryReader(json_credentials_path='/path/to/creds.json')

HiveReader

Coming soon...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for metric-builder, version 0.0.3
Filename, size File type Python version Upload date Hashes
Filename, size metric_builder-0.0.3-py3-none-any.whl (8.5 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size metric_builder-0.0.3.tar.gz (3.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page