Skip to main content

ruia_ua - simple user-agent middleware for Ruia.

Project description

ruia-ua

simple user-agent middleware for Ruia

Installation

pip install -U ruia-ua

Usage

ruia-ua will be automatically with a random User-Agent for your request's headers

from ruia import AttrField, TextField, Item, Spider
from ruia_ua import middleware


class HackerNewsItem(Item):
    target_item = TextField(css_select='tr.athing')
    title = TextField(css_select='a.storylink')
    url = AttrField(css_select='a.storylink', attr='href')


class HackerNewsSpider(Spider):
    start_urls = ['https://news.ycombinator.com/news?p=1', 'https://news.ycombinator.com/news?p=2']

    async def parse(self, response):
        # Do something...
        print(response.html)


if __name__ == '__main__':
    HackerNewsSpider.start(middleware=middleware)

Enjoy it :)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ruia-ua, version 0.0.2
Filename, size File type Python version Upload date Hashes
Filename, size ruia_ua-0.0.2-py3-none-any.whl (11.5 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size ruia_ua-0.0.2.tar.gz (11.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page