super fast web crawler
Note: this package is still in beta. Use with caution !
Quickstart
Install netboy:
pip install netboy
Use netboy:
if __name__ == '__main__':
setup_log('netboy')
data = [ 'http://www.bing.com',]
boy = NetBoy()
boy.use_queue(
'worker'
).use_spider(
'pycurl'
).use_filter(
['url', 'title', 'code']
).use_workers().use_triggers([
'test.functional.netboy.screen.print_screen'
])
resp = boy.run(data)1 maintainer
Metadata
1 maintainer
Release files for netboy 2018.8.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| netboy-2018.8.7-py3-none-any.whl | Python 3 | none | any | Details |
Release files / netboy-2018.8.7-py3-none-any.whl
| Download URL | netboy-2018.8.7-py3-none-any.whl |
|---|---|
| Size | 3.9 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f7603294b2353a6efb17367faa8df2f348230d1362b472a0464c20d4f4e8126e
|
|
BLAKE2b-256 checksum How to use checksums |
5c59a6b37dacef0c54f2e1da06c035b56ba9bffafa0425c62fd2a1e013f35b87
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
Python-urllib/3.6
|
