Skip to main content

A random user-agent for all your needs

Project description

scrapy-random-useragent-pro

Random UserAgent middleware for Scrapy.

You can chose your preferred ua in request.meta[desktop,mobile,bot].

Install

pip install scrapy-random-useragent-pro

Usage

settings.py

RANDOM_UA_ENABLED = True
RANDOM_UA_DEFAULT_TYPE = 'mobile'
# always change user-agent
RANDOM_UA_OVERWRITE = False


DOWNLOADER_MIDDLEWARES = {
    'scrapy_random_useragent_pro.middleware.RandomUserAgentMiddleware': 100,
    'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None,# important
}

In your code

yield Request('http://scrapytest.org/', meta={'ua': 'mobile'})

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy-random-useragent-pro-1.0.0.tar.gz (3.9 kB view hashes)

Uploaded source

Built Distribution

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page