Skip to main content

Proxy Port Scrapy middleware

Project description

Proxy Port integration for Scrapy. About Proxy Port

Prerequisites

To use this package you will need a free API key. Get your API key here. Detailed instructions here.

Installation

$ pip install scrapyproxyport

Getting Started

You need to assign an API key. This can be done either through an environment variable

$ export PROXY_PORT_API_KEY=<API_KEY>

or inside settings.py.

# inside <your_project>/settings.py

PROXY_PORT_API_KEY = '<API_KEY>'
DOWNLOADER_MIDDLEWARES = {
    # Add middleware with order number right before CookiesMiddleware
    'scrapyproxyport.middlewares.ProxyMiddleware': 898,
    'scrapy.downloadermiddlewares.cookies.CookiesMiddleware': 899,
}
DOWNLOAD_TIMEOUT = 10
RETRY_TIMES = 20
DUPEFILTER_CLASS = 'scrapy.dupefilters.BaseDupeFilter'

Example Scrapy project

scrapy-proxyport-example

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapyproxyport-1.1.1.tar.gz (4.7 kB view details)

Uploaded Source

Built Distribution

scrapyproxyport-1.1.1-py3-none-any.whl (5.2 kB view details)

Uploaded Python 3

File details

Details for the file scrapyproxyport-1.1.1.tar.gz.

File metadata

  • Download URL: scrapyproxyport-1.1.1.tar.gz
  • Upload date:
  • Size: 4.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.9

File hashes

Hashes for scrapyproxyport-1.1.1.tar.gz
Algorithm Hash digest
SHA256 1be9633479c04ca34aa074aefa1fad06fb39b206006f1072e4814cad5b5a4347
MD5 c1363ea47b4fd4ec5e7704afe18ec3ae
BLAKE2b-256 d1f9068500a949f86849a1d8051bb85990023df82a6d94a20e9562968c3d2156

See more details on using hashes here.

Provenance

File details

Details for the file scrapyproxyport-1.1.1-py3-none-any.whl.

File metadata

File hashes

Hashes for scrapyproxyport-1.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 351899ab71c67f4025124c7f17b1653758ceb3a4425fbff0559f7b677a2f28bf
MD5 5c5badfcf9f9a8147a8c8d87da35cd39
BLAKE2b-256 ee0a4e45b4a67aecbeb8c5bdd798349b27c8f624d11fd3480acdf73fb4c3cf5e

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page