Skip to main content

Executes migrated scrapy project to Apify platform

Project description

Apify Scrapy Executor

Support module for Apify Scrapy Migrator. Runs migrated Scrapy project and uploads result to Apify default dataset.

Usage

Since apify-client does not fully support local runs yet, migrated Scrapy projects work best on Apify Cloud. Apify Scrapy Migrator is recommended to be used with Apify Scrapy Migrator on Apify Cloud.

But for people who want to earn their results, here is a small example.

# import SpiderExecutor class
from apify_scrapy_executor import SpiderExecutor
# create instance of the Spider executor class with scrapy class as an argument
spider_executor = SpiderExecutor(scrapy_class)
# run the scrapy project with dataset_id argument to save the result and args_dict for scrapy input
spider_executor.run(dataset_id=apify_dataset, args_dict=actor_input)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

apify_scrapy_executor-0.0.10.tar.gz (6.5 kB view hashes)

Uploaded Source

Built Distribution

apify_scrapy_executor-0.0.10-py3-none-any.whl (7.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page