Skip to main content

Executes migrated scrapy project to Apify platform

Project description

Apify Scrapy Executor

Support module for Apify Scrapy Migrator. Runs migrated Scrapy project and uploads result to Apify default dataset.

Usage

Since apify-client does not fully support local runs yet, migrated Scrapy projects work best on Apify Cloud. Apify Scrapy Migrator is recommended to be used with Apify Scrapy Migrator on Apify Cloud.

But for people who want to earn their results, here is a small example.

# import SpiderExecutor class
from apify_scrapy_executor import SpiderExecutor
# create instance of the Spider executor class with scrapy class as an argument
spider_executor = SpiderExecutor(scrapy_class)
# run the scrapy project with dataset_id argument to save the result and args_dict for scrapy input
spider_executor.run(dataset_id=apify_dataset, args_dict=actor_input)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

apify_scrapy_executor-0.0.10.tar.gz (6.5 kB view details)

Uploaded Source

Built Distribution

apify_scrapy_executor-0.0.10-py3-none-any.whl (7.1 kB view details)

Uploaded Python 3

File details

Details for the file apify_scrapy_executor-0.0.10.tar.gz.

File metadata

File hashes

Hashes for apify_scrapy_executor-0.0.10.tar.gz
Algorithm Hash digest
SHA256 f201161cdd5c74a8e557e075004de9e2a6026f2e57bea28426d7c81fda205514
MD5 60174bb448d326c0822ccfb7564bc617
BLAKE2b-256 28dd0a7e77362cecea56fe761ce0d608ef08e94d945ddb510e6d977294a453e7

See more details on using hashes here.

File details

Details for the file apify_scrapy_executor-0.0.10-py3-none-any.whl.

File metadata

File hashes

Hashes for apify_scrapy_executor-0.0.10-py3-none-any.whl
Algorithm Hash digest
SHA256 fd98bf19506bcd1afd61d8a03ed9a3e21284d9e90c27a98d5eff8267f2012a85
MD5 c51d8f95aca002da42f0673e34006f40
BLAKE2b-256 6dd4dc37c941a6aa8559be86e6d73fc5326b001286532f5279992eb4ac41e6db

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page