Scrapinghub Hubstorage Collection scanner.
Project description
Provides convenient way to scan a collection in batches
Allows to merge data from multiple collections
Accepts endts and startts in many string formats (as accepted by dateparser lib) or standard HS epoch in millisecs
Accepts excluded prefixes
Adds stopbefore feature (analogous to startafter but the inverse)
Provides method for arbitrary prefix aggregation counting
Supports partitioned collections
Provides a suite for testing hs collection code.
Up to version 0.1.6: Python2 only Starting version 0.2: Python3 only
See usage instructions at scanner.py docstring.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file collection_scanner-0.3.tar.gz.
File metadata
- Download URL: collection_scanner-0.3.tar.gz
- Upload date:
- Size: 7.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.6.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | a4c31788a8860c904e5f8505b59ac97c3c1a915b73db4de7cd31ac24e71d9d8d |
|
| MD5 | 92ce3b33d8cb8c948334737c1664ebf5 |
|
| BLAKE2b-256 | 6e42b347735c75b411d577b92841f22ff20b957635f67ff16248122917ff5e04 |
File details
Details for the file collection_scanner-0.3-py3-none-any.whl.
File metadata
- Download URL: collection_scanner-0.3-py3-none-any.whl
- Upload date:
- Size: 10.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.26.0 CPython/3.6.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 1511518094ae71fa6921d7ebcd733388841ab6aff069de2091bc4351651b60f1 |
|
| MD5 | 941d951762e2aa21d6eac2e72c866dba |
|
| BLAKE2b-256 | 3618d264b940153b49ae7b37bbee7b4aa1b0a716ae166c24bc1663ae1725ab0b |
