Skip to main content

Locally download a datamining dataset from the Eulerian Technologies API

Project description

eanalytics_api_py

A python 3 module to retrieve data from Eulerian Technologies API

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

Having a python 3 environment

Installing

pip3 install eanalytics_api_py

Upgrading

pip3 install eanalytics_api_py --upgrade

Running

Connexion class

from  eanalytics_api_py.conn import Conn
from  eanalytics_api_py.datamining_helper import deduplicate_product_cols_file_2_df
import pandas as pd

language = 'en'

conn = Conn(
        gridpool_name='demo',
        datacenter='com',
        api_key='key',
        print_log = True
)

download_datamining method

path2file = conn.download_datamining(
                website_name = 'demo',
                datamining_type = 'order',
                payload = {
                    'date-from':'05/01/2020',
                    'date-to':'05/01/2020',
                    'with-ordertype':1,
                    'with-mdevicetype' : 1,
                    'with-last-channel':1,
                    'with-cgiparam':1,
                    'with-channel-count':1,
                    'with-channel-level':1
                    'with-orderproduct' : 1,
                    'with-productparam' : 1,
                    'with-productgroup' : 1,
                    'ea-lg' : language,
                },
                output_filename = 'demo_orders.csv.gzip',
                output_directory = '',
                override_file=True
)

# load into pandas dataframe and normalize product columns
df = deduplicate_product_cols_file_2_df(
        path2file,
        language=language
)
  • Use this doc to customize your payload object:

download_edw method

query = """ GET {
  TIMERANGE { 1571912419 1571936095 }
  READERS {
    ea:order@demo AS order
  }
  OUTPUTS_ROW( order ) {
    order.uid, order.timestamp, order.orderref, order.orderstatus, order.amount
  }
};"""


path2file = conn.download_edw(
            query=query,
            ip='ip1,ip2,ip3',
            token_path2file = 'edw_token.json',
            force_token_refresh = False,
            output_filename = 'edw_test.csv.gzip',
            output_directory='',
           # jobrun_id = 0,
            override_file=True,
)

Eulerian Datawharehouse documentation TODO: query builder doc

Notes

  • The file downloaded is gzipped
  • Explore your data using jupyter-notebook along with pandas and seaborn for plug'n'play data cleaning/visualisation
  • Samples jupyter notebooks featuring different use-cases will be uploaded in the near future
  • A docker image containing jupyter and our librairy is available here

Author

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

eanalytics_api_py-0.0.69.tar.gz (8.4 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page