watchdogs 
Keep an eye on the change of web world.
Such as post articles / news on the web portal / server api health / binge-watching / steam price fluctuation / github events / updates of comic and novel, and so on...
Intro
- This is a web app based on fastapi, databases, uniparser, torequests.
- Smoothly deploy it by pip:
pip install -U watchdogs;python3 -m watchdogs - Simple to create a new crawler with the Web UI, not like old ways to write duplicate code.
- All the crawlers keep runing in the async environment.
- Almost all the elements have a title attribute to describe the features in the Web UI, which means docs lay on the UI.
- Release your hands from repetitive refreshing pages on the browser.
- Subscribe the change events with RSS reminder extensions, such as Feedbro or RSS Feed Reader.
- Implement a class which inherits from
watchdogs.callbacks.Callback.
Usage
-
pip install -U watchdogs
-
python -m watchdogs
-
Open the browser: http://127.0.0.1:9901
Command line args
python -m watchdogs -- -h
- db_url:
sqlite / mysql / postgresql(not test) url, which databases supports. Defaults to 'sqlite:///{HOME_PATH}/watchdogs/storage.sqlite'
- password:
init password, if null can be set on the first visit on web.
- mute_std_log:
remove stdout log for clean stream
- mute_file_log:
ignore file log located at {HOME_PATH}/watchdogs folder.
- md5_salt:
md5_salt for custom md5(password) / md5(rss_tag)
- config_dir:
config dir to save the logs and config files, if using sqlite include sqlite file. defaults to {HOME_PATH}/watchdogs
- use_default_cdn:
If Config.cdn_urls not set, and use_default_cdn is True, will use online js/css cdn links from staticfile.org.
- **uvicorn_kwargs:
uvicorn startup kwargs, such as port, host. Which can be set like:
python -m watchdogs --port=9999 --host=127.0.0.1 --access-log=False
Quick Start to Create New Task
Web UI
Screenshots
- Welcome Page (Tasks Page).
Here you can see all the tasks meta, goto RSS / Mobile Lite Page, and do some operations to the tasks.
- New Task Page.
Here based on the latest uniparser web app, to create new rules and also tasks.
- Rules Page.
Do some operations for the rules.
- API page.
Based on fastapi
/docswhich is generated automatically.
- Mobile Page (Lite View).
For mobile phone to glimpse the latest result for the current 30 tasks.
1 maintainer
Metadata
1 maintainer
Release files for watchdogs 1.6.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| watchdogs-1.6.7-py3-none-any.whl | Python 3 | none | any | Details |
Release files / watchdogs-1.6.7-py3-none-any.whl
| Download URL | watchdogs-1.6.7-py3-none-any.whl |
|---|---|
| Size | 302.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0ad5885708324b21c6cc8ea3a155af85283a54ab996117eabce38111c0bf0ff0
|
|
BLAKE2b-256 checksum How to use checksums |
30eda0604de6acee7cae3217f7f6e6ee6120637adc94b495e43144837071ba14
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.4.2 requests/2.23.0 setuptools/41.0.1 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.7.1
|
