Skip to main content

Common utilities for EQI project.

Project description

EQI Data Loader and Exporter

Features:

  • Download DataFrame from data vendors via S3
  • Manage and store analysis views locally or remotely on S3
  • Download DataFrame from RESLIVE DB

Key Concept:

  • Bundle: a group of data files from a specific vendor, e.g. linkup
  • DataFile: a file which contains data, e.g. Company_Daily within linkup bundle
  • View: a DataFrame serialized into a parquet file, can be stored locally, or remotely on S3
  • DBView: a DataFrame which contains the result of a SQL query

How to install:

  • Install this package using pip

    pip install eqi-utilities
    

Getting started:

  • Create EQI home folder
    • Create a folder named ‘.eqi’ in your home folder, on Windows, create a folder named as ‘.eqi.’
  • Go to examples/config.ini, set all the fields, and copy the config.ini file to EQI home folder created in the above step
  • Go to examples/load_export_example.html, see how to use the utilities

Known Issues:

  • Failed at downloading large parquet datafile

    Please install fastparquet using conda: .. code:: bash

    conda install -c conda-forge fastparquet

Please feel free to create an issue in case of any problems.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for eqi-utils, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size eqi_utils-1.0.1-py3-none-any.whl (13.9 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size eqi-utils-1.0.1.tar.gz (25.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page