help you build a small spider quickly.
Project description
# QuickSpider
Thid is a samll spider framework which based on nodes and graph built with these nodes.
The goal of this samll framework is to build a quick spider.
## Install
use pip
`bash pip install -i https://test.pypi.org/simple/ quickspider `
## How to use it?
custome your nodes. (or use nodes writen by others or builtin nodes)
build graph using the nodes. (use toml file)
run it.
## TODO
[ ] add node operation like node combination
[ ] parser to define the nodes and construct the graph. (maybe use syntax like markdown)
[ ] add more parse method like xpath, regex…
[ ] add logger system
[ ] try to add asyn feature
[ ] robust
[ ] some useful nodes for spider builders
[ ] maybe a market place to share your nodes.
[ ] add gui editor
[ ] more and more…
## Example for current version
`bash quickspider create --template default `
then, you can find one toml file: default.toml
`bash quickspider run --file deafult.toml `
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for quickspider-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6900da6a37999940d71a10ba205232fbf0bcc84106bfcf49444dba9e212b09a3 |
|
MD5 | b227d3c7bc696f47559e6bd53a633b8d |
|
BLAKE2b-256 | 9c507ec88bd30a33b7d314c67d23523aa8ea1cb9cebaab08ad745b82619afd88 |