Skip to main content

ruia_ua - simple user-agent middleware for Ruia.

Project description

ruia-ua

simple user-agent middleware for Ruia

Installation

pip install -U ruia-ua

Usage

ruia-ua will be automatically with a random User-Agent for your request's headers

from ruia import AttrField, TextField, Item, Spider
from ruia_ua import middleware


class HackerNewsItem(Item):
    target_item = TextField(css_select='tr.athing')
    title = TextField(css_select='a.storylink')
    url = AttrField(css_select='a.storylink', attr='href')


class HackerNewsSpider(Spider):
    start_urls = ['https://news.ycombinator.com/news?p=1', 'https://news.ycombinator.com/news?p=2']

    async def parse(self, response):
        # Do something...
        print(response.html)


if __name__ == '__main__':
    HackerNewsSpider.start(middleware=middleware)

Enjoy it :)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ruia-ua, version 0.0.2
Filename, size File type Python version Upload date Hashes
Filename, size ruia_ua-0.0.2-py3-none-any.whl (11.5 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size ruia_ua-0.0.2.tar.gz (11.3 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page