Compare Files provides a way to compare latency and count products, between two systems with incoming data. In this case, we are comparing the ingested products on the ESB between legacy and enterprise. A file from the legacy system is is downloaded via GoogleDrive and the enterprise file is created from querying the edpdb database. Both files are CSV.
Typical usage often looks like this:
python database_query.py python compare_files.py
Input
Dates for querying the database in the format MM-DD-YYYY
Name of datafile from legacy system
Prerequisites
Connection to the current host: {username}@nco-lw-{username}.ncep.noaa.gov
Python 2.7
Download legacy file from GoogleDrive: https://drive.google.com/open?id=1dXkpZ4LxmS_6Sah7GheAnwAZmj7727Fp
Download AWDS file from GoogleDrive: https://drive.google.com/open?id=1NP6d3GnNdw4UoWF7-aJlSy8saKwE9sBW
Install module psycopg2 http://initd.org/psycopg/
Other modules needed: csv, datetime, re, and time
Output
Latency.csv
LegacyProductsNotInEnterprise.csv
Troubleshooting
KeyError or ValueError (Product Comparison): There is likely an error in the legacy data file. Sometimes this is something as simple as the inclusion/exclusion of AM or PM in the timestamp or a leading comma in a column.
Missing Products (order of 500+): The database is missing data for a date due to an outage, restart, etc. The AWDS table file is not up to date. The date range is incorrect.
2 maintainers
Metadata
2 maintainers
Release files for emwinCompareFiles 0.2.11
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| emwinCompareFiles-0.2.11.tar.gz | 6.2 kB | Without provenance | Details for emwinCompareFiles-0.2.11.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| emwinCompareFiles-0.2.11-py2-none-any.whl | Python 2 | none | any | Details for emwinCompareFiles-0.2.11-py2-none-any.whl |
Total release size: 13.5 kB
Release files / emwinCompareFiles-0.2.11.tar.gz
| Download URL | emwinCompareFiles-0.2.11.tar.gz |
|---|---|
| Size | 6.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
76ba5209ecfd6920e1a66611c5169de42d285351629fc018a3ac239f12157026
|
|
BLAKE2b-256 checksum How to use checksums |
1358f7f0b4e15bbb0a147ab41da6873fab809653fdbae54bb2127aabeec1c728
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/2.7.5
|
Release files / emwinCompareFiles-0.2.11-py2-none-any.whl
| Download URL | emwinCompareFiles-0.2.11-py2-none-any.whl |
|---|---|
| Size | 7.3 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
668e7603ea85a948c8bf5dacd6541386adbc47d9b7e8533c8c3eab801627e724
|
|
BLAKE2b-256 checksum How to use checksums |
f0b364d77990b67910144e0e6519ea544e86ba356c50458b5556d483deb0f5f9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/2.7.5
|
Security for emwinCompareFiles 0.2.11
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
