Crawler integration with INSPIRE-HEP.
Project description
Crawler integration with INSPIRE-HEP using scrapy project HEPCrawl.
This module allows scheduling of crawler jobs to a Scrapyd instance serving a Scrapy project. E.g. in this case the default scrapy project is HEPCrawl.
It integrates directly with invenio-workflows module to create workflows for every record harvested by the crawler.
This module is meant to use only with INSPIRE-HEP overlay. Use at own risk.
Full documentation is hosted here: http://pythonhosted.org/inspire-crawler/
See also documentation of HEPCrawl: http://pythonhosted.org/hepcrawl/
Changes
Version 0.1.0 (released 2016-06-02)
Initial public release.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file inspire-crawler-0.1.0.tar.gz.
File metadata
- Download URL: inspire-crawler-0.1.0.tar.gz
- Upload date:
- Size: 29.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 54eefafa3d71e1d30844f4079be2a9dd31a58f994f28308f72c4c05b5a3fe57c |
|
| MD5 | c9222706fa66588b817866c564923474 |
|
| BLAKE2b-256 | 41626269f8f4782e52f13ddca0c74ae3242b20d4a00ae42179c42833784dc37e |
