geo-spider
crawl all GEO metadata, features:
- crawl platforms
- crawl samples
- crawl series
- incremental crawling
- missed crawling
Table of Contents
installation
pip install geo-spider
output file format
geo-spider saves files in jsonlines form, Refer to this site for details.
logs
geo-spider default generate logs to geo-spider.log(current directory)
in WARNING level, you can customize by -d and -l options.
-dto enable debug mode-lspecify customized log file
geo-spider -d -l new-geo-spider.log <sub-command>
platforms
platforms denovo crawling
geo-spider platforms -o platforms.jl
platforms incremental crawling
If you have a crawled platforms jsonlines file:
geo-spider platforms -cf platforms.jl -o new-platforms.jl
If you have multiple platforms jsonlines files:
geo-spider platforms -cd platforms -o new-platforms.jl
platforms missed crawling
Specify -cf or -cd like incremental crawling, add a -m option.
geo-spider platforms -cf platforms.jl -m missed -o new-platforms.jl
samples
samples denovo crawling
geo-spider samples -o samples.jl
samples incremental crawling
geo-spider samples -pcf platforms.jl -cf samples.jl -o new-samples.jl
samples missed crawling
geo-spider samples -pcf platforms.jl -cf samples.jl -m missed -o new-samples.jl
series
series denovo crawling
geo-spider series -o series.jl
series incremental crawling
geo-spider series -pcf platforms.jl -scf samples.jl -cf series.jl -o new-series.jl
series missed crawling
geo-spider series -pcf platforms.jl -scf samples.jl -cf series.jl -m missed -o new-series.jl
1 maintainer
Metadata
1 maintainer
Release files for geo-spider 0.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| geo-spider-0.0.5.tar.gz | 5.3 kB | Without provenance | Details for geo-spider-0.0.5.tar.gz |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| geo_spider-0.0.5-py3.7.egg | Legacy Egg format | - | - | Details for geo_spider-0.0.5-py3.7.egg |
| geo_spider-0.0.5-py3-none-any.whl | Python 3 | none | any | Details for geo_spider-0.0.5-py3-none-any.whl |
Total release size: 19.3 kB
Release files / geo-spider-0.0.5.tar.gz
| Download URL | geo-spider-0.0.5.tar.gz |
|---|---|
| Size | 5.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d7f72efeb3eb0f0c47e2769b687532a3bc05224fe71736b517a99c60fa4f8205
|
|
BLAKE2b-256 checksum How to use checksums |
e60411427d6fb07b376428e28e259e75dcacf406df624d803bc3ecec0084d28a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.4.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.7.7
|
Release files / geo_spider-0.0.5-py3.7.egg
| Download URL | geo_spider-0.0.5-py3.7.egg |
|---|---|
| Size | 9.2 kB |
| Tags | Egg |
|
SHA-256 checksum How to use checksums |
94fc97203fe489df737e85bf2d6339f56450fcbc217f2c1c231c221e2e6b87e5
|
|
BLAKE2b-256 checksum How to use checksums |
d9a0dbfe5612194751ce973c5688855c4b8c520e5a46709bc91ae28871c28cfa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.4.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.7.7
|
Release files / geo_spider-0.0.5-py3-none-any.whl
| Download URL | geo_spider-0.0.5-py3-none-any.whl |
|---|---|
| Size | 4.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9dabb06e1a2ca91f4e6c052554070da5746551cf414d7c9af5d9120ee60e2fb6
|
|
BLAKE2b-256 checksum How to use checksums |
fd273ea9ae0aba2fae8212c6d0756953cdeb8b4f93c399bbded6a7733cce9d1a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.4.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.7.7
|
Security for geo-spider 0.0.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
