Skip to main content

ruia_ua - simple user-agent middleware for Ruia.

Project description

## ruia-ua

simple user-agent middleware for [ruia](https://github.com/howie6879/ruia)

```text
Notice: Works on ruia >= 0.0.1
```

### Installation

```shell
pip install ruia-ua
```

### Usage

`ruia-ua` will be automatically populated with a random `User-Agent` for your request's `headers`

```python
from ruia import AttrField, TextField, Item, Spider
from ruia import middleware


class HackerNewsItem(Item):
target_item = TextField(css_select='tr.athing')
title = TextField(css_select='a.storylink')
url = AttrField(css_select='a.storylink', attr='href')

async def clean_title(self, value):
return value


class HackerNewsSpider(Spider):
start_urls = ['https://news.ycombinator.com/news?p=1', 'https://news.ycombinator.com/news?p=2']
concurrency = 10

async def parse(self, res):
items = await HackerNewsItem.get_items(html=res.html)
for item in items:
print(item.title)


if __name__ == '__main__':
HackerNewsSpider.start(middleware=middleware)
```

Enjoy it :)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ruia_ua-0.0.1.tar.gz (11.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ruia_ua-0.0.1-py3-none-any.whl (11.6 kB view details)

Uploaded Python 3

File details

Details for the file ruia_ua-0.0.1.tar.gz.

File metadata

  • Download URL: ruia_ua-0.0.1.tar.gz
  • Upload date:
  • Size: 11.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.27.0 CPython/3.6.5

File hashes

Hashes for ruia_ua-0.0.1.tar.gz
Algorithm Hash digest
SHA256 331c42e86c510da96cbcda79aa3f06cb27635f80f1c8f2d4f4ab98f2968ef346
MD5 70f0d75503d6fa72eae989b703aa780a
BLAKE2b-256 24b34a2b235ad349b3fe22255b379b28bceee57c2f1831c44c1ebe85877f99b9

See more details on using hashes here.

File details

Details for the file ruia_ua-0.0.1-py3-none-any.whl.

File metadata

  • Download URL: ruia_ua-0.0.1-py3-none-any.whl
  • Upload date:
  • Size: 11.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.27.0 CPython/3.6.5

File hashes

Hashes for ruia_ua-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 935e678e3dbd0beea8b776351effcae03ee0f3dc5fff7988623cd8a71412d2b5
MD5 74b8877a04f483336d9ee20bcec47cc3
BLAKE2b-256 bd05ff83afe1b9cd9bf92c8cf6510f5e1f123cd99d3c0671077d06f1db61129d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page