ruia_ua - simple user-agent middleware for Ruia.
Project description
## ruia-ua
simple user-agent middleware for [ruia](https://github.com/howie6879/ruia)
```text
Notice: Works on ruia >= 0.0.1
```
### Installation
```shell
pip install ruia-ua
```
### Usage
`ruia-ua` will be automatically populated with a random `User-Agent` for your request's `headers`
```python
from ruia import AttrField, TextField, Item, Spider
from ruia import middleware
class HackerNewsItem(Item):
target_item = TextField(css_select='tr.athing')
title = TextField(css_select='a.storylink')
url = AttrField(css_select='a.storylink', attr='href')
async def clean_title(self, value):
return value
class HackerNewsSpider(Spider):
start_urls = ['https://news.ycombinator.com/news?p=1', 'https://news.ycombinator.com/news?p=2']
concurrency = 10
async def parse(self, res):
items = await HackerNewsItem.get_items(html=res.html)
for item in items:
print(item.title)
if __name__ == '__main__':
HackerNewsSpider.start(middleware=middleware)
```
Enjoy it :)
simple user-agent middleware for [ruia](https://github.com/howie6879/ruia)
```text
Notice: Works on ruia >= 0.0.1
```
### Installation
```shell
pip install ruia-ua
```
### Usage
`ruia-ua` will be automatically populated with a random `User-Agent` for your request's `headers`
```python
from ruia import AttrField, TextField, Item, Spider
from ruia import middleware
class HackerNewsItem(Item):
target_item = TextField(css_select='tr.athing')
title = TextField(css_select='a.storylink')
url = AttrField(css_select='a.storylink', attr='href')
async def clean_title(self, value):
return value
class HackerNewsSpider(Spider):
start_urls = ['https://news.ycombinator.com/news?p=1', 'https://news.ycombinator.com/news?p=2']
concurrency = 10
async def parse(self, res):
items = await HackerNewsItem.get_items(html=res.html)
for item in items:
print(item.title)
if __name__ == '__main__':
HackerNewsSpider.start(middleware=middleware)
```
Enjoy it :)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ruia_ua-0.0.1.tar.gz
(11.4 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
ruia_ua-0.0.1-py3-none-any.whl
(11.6 kB
view details)
File details
Details for the file ruia_ua-0.0.1.tar.gz.
File metadata
- Download URL: ruia_ua-0.0.1.tar.gz
- Upload date:
- Size: 11.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.27.0 CPython/3.6.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
331c42e86c510da96cbcda79aa3f06cb27635f80f1c8f2d4f4ab98f2968ef346
|
|
| MD5 |
70f0d75503d6fa72eae989b703aa780a
|
|
| BLAKE2b-256 |
24b34a2b235ad349b3fe22255b379b28bceee57c2f1831c44c1ebe85877f99b9
|
File details
Details for the file ruia_ua-0.0.1-py3-none-any.whl.
File metadata
- Download URL: ruia_ua-0.0.1-py3-none-any.whl
- Upload date:
- Size: 11.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.27.0 CPython/3.6.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
935e678e3dbd0beea8b776351effcae03ee0f3dc5fff7988623cd8a71412d2b5
|
|
| MD5 |
74b8877a04f483336d9ee20bcec47cc3
|
|
| BLAKE2b-256 |
bd05ff83afe1b9cd9bf92c8cf6510f5e1f123cd99d3c0671077d06f1db61129d
|