Executes migrated scrapy project to Apify platform
Project description
Apify Scrapy Executor
Support module for Apify Scrapy Migrator. Runs migrated Scrapy project and uploads result to Apify default dataset.
Usage
Since apify-client
does not fully support local runs yet, migrated
Scrapy projects work best on Apify Cloud. Apify Scrapy Migrator is recommended to be used with Apify Scrapy Migrator on
Apify Cloud.
But for people who want to earn their results, here is a small example.
# import SpiderExecutor class
from apify_scrapy_executor import SpiderExecutor
# create instance of the Spider executor class with scrapy class as an argument
spider_executor = SpiderExecutor(scrapy_class)
# run the scrapy project with dataset_id argument to save the result and args_dict for scrapy input
spider_executor.run(dataset_id=apify_dataset, args_dict=actor_input)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file apify_scrapy_executor-0.0.10.tar.gz
.
File metadata
- Download URL: apify_scrapy_executor-0.0.10.tar.gz
- Upload date:
- Size: 6.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.0 CPython/3.9.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f201161cdd5c74a8e557e075004de9e2a6026f2e57bea28426d7c81fda205514 |
|
MD5 | 60174bb448d326c0822ccfb7564bc617 |
|
BLAKE2b-256 | 28dd0a7e77362cecea56fe761ce0d608ef08e94d945ddb510e6d977294a453e7 |
File details
Details for the file apify_scrapy_executor-0.0.10-py3-none-any.whl
.
File metadata
- Download URL: apify_scrapy_executor-0.0.10-py3-none-any.whl
- Upload date:
- Size: 7.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.0 CPython/3.9.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd98bf19506bcd1afd61d8a03ed9a3e21284d9e90c27a98d5eff8267f2012a85 |
|
MD5 | c51d8f95aca002da42f0673e34006f40 |
|
BLAKE2b-256 | 6dd4dc37c941a6aa8559be86e6d73fc5326b001286532f5279992eb4ac41e6db |