Scraping and Web Crawling Framework
Project description
## Getting started
```python
❯ git clone https://github.com/dongweiming/daenerys
❯ cd daenerys
❯ virtualenv venv
❯ source venv/bin/activate
❯ pip install -r requirements.txt
# 安装配置Redis和MongoDB
❯ cd demo
❯ python worker.py # 启动worker(默认5个进程)
❯ python beat.py # 新开启一个终端，启动Beat服务生成任务
❯ ipython # 发布一个需要获取执行结果的任务
In [1]: from messaging import sync_get
In [2]: sync_get('flask')
PUT flask
Out[2]:
{u'author': u'Armin Ronacher',
u'download_url': u'https://pypi.python.org/packages/24/6e/11b9c57e46f276a8a8da85a2fa7ada62b0463b68693616c7ab5df356fa/Flask-0.12.1.tar.gz',
u'name': u'flask',
u'version': u'0.12.1'}
```
```python
❯ git clone https://github.com/dongweiming/daenerys
❯ cd daenerys
❯ virtualenv venv
❯ source venv/bin/activate
❯ pip install -r requirements.txt
# 安装配置Redis和MongoDB
❯ cd demo
❯ python worker.py # 启动worker(默认5个进程)
❯ python beat.py # 新开启一个终端，启动Beat服务生成任务
❯ ipython # 发布一个需要获取执行结果的任务
In [1]: from messaging import sync_get
In [2]: sync_get('flask')
PUT flask
Out[2]:
{u'author': u'Armin Ronacher',
u'download_url': u'https://pypi.python.org/packages/24/6e/11b9c57e46f276a8a8da85a2fa7ada62b0463b68693616c7ab5df356fa/Flask-0.12.1.tar.gz',
u'name': u'flask',
u'version': u'0.12.1'}
```
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
daenerys-0.1.tar.gz
(4.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file daenerys-0.1.tar.gz.
File metadata
- Download URL: daenerys-0.1.tar.gz
- Upload date:
- Size: 4.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.13.0 setuptools/28.8.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.6.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d64b2e0db7a5993ca1026e92b773d465ff3261ae300383f98e0461af3caa1597
|
|
| MD5 |
d639c20c8ecf23d75001440cc6938b4e
|
|
| BLAKE2b-256 |
aecd28295e5b88684668696200e671ed59cba23d94c2da85a34ad24543686f84
|
File details
Details for the file daenerys-0.1-py3-none-any.whl.
File metadata
- Download URL: daenerys-0.1-py3-none-any.whl
- Upload date:
- Size: 7.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.13.0 setuptools/28.8.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.6.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b2924f5dd482fcfef9738fea200988151726e36354a4ff21f86fc333492e62e6
|
|
| MD5 |
72f859d37078782b15cb25ad536c0614
|
|
| BLAKE2b-256 |
1f498ba99f4b17b29fb724860644ee1217fea2286331c574e85d640843827db2
|
