EMWIN Latency Comparison.
Project description
Compare Files provides a way to compare latency and count products, between two systems with incoming data. In this case, we are comparing the ingested products on the ESB between legacy and enterprise. A file from the legacy system is is downloaded via GoogleDrive and the enterprise file is created from querying the edpdb database. Both files are CSV.
Typical usage often looks like this:
python database_query.py python compare_files.py
Input
Dates for querying the database in the format MM-DD-YYYY
Name of datafile from legacy system
Prerequisites
Connection to the current host: {username}@nco-lw-{username}.ncep.noaa.gov
Python 2.7
Download legacy file from GoogleDrive: https://drive.google.com/open?id=1dXkpZ4LxmS_6Sah7GheAnwAZmj7727Fp
Download AWDS file from GoogleDrive: https://drive.google.com/open?id=1NP6d3GnNdw4UoWF7-aJlSy8saKwE9sBW
Install module psycopg2 http://initd.org/psycopg/
Other modules needed: csv, datetime, re, and time
Output
Latency.csv
LegacyProductsNotInEnterprise.csv
Troubleshooting
KeyError or ValueError (Product Comparison): There is likely an error in the legacy data file. Sometimes this is something as simple as the inclusion/exclusion of AM or PM in the timestamp or a leading comma in a column.
Missing Products (order of 500+): The database is missing data for a date due to an outage, restart, etc. The AWDS table file is not up to date. The date range is incorrect.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for emwinCompareFiles-0.2.11-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 668e7603ea85a948c8bf5dacd6541386adbc47d9b7e8533c8c3eab801627e724 |
|
MD5 | f1efe283a2341772145ca21548919e84 |
|
BLAKE2b-256 | f0b364d77990b67910144e0e6519ea544e86ba356c50458b5556d483deb0f5f9 |