Watchdogs to keep an eye on the world's change. Read more: https://github.com/ClericPy/watchdogs.
Project description
watchdogs ![PyPI](https://pypi-camo.freetls.fastly.net/5ae4d864c7786770adb7faa0f7bb92aef001e8a4/68747470733a2f2f696d672e736869656c64732e696f2f707970692f762f7761746368646f67733f7374796c653d706c6173746963)
![PyPI - Wheel](https://pypi-camo.freetls.fastly.net/9db7dd6b6504607831b77fb3c5b8d63fbff5c80d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f776865656c2f7761746368646f67733f7374796c653d706c6173746963)
![PyPI - Python Version](https://pypi-camo.freetls.fastly.net/6e6156a26c5b50aa4329942753f8857b64e2e38d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f707976657273696f6e732f7761746368646f67733f7374796c653d706c6173746963)
![PyPI - Downloads](https://pypi-camo.freetls.fastly.net/9494177b549597acea3e8b4c300b83130d4a87e0/68747470733a2f2f696d672e736869656c64732e696f2f707970692f646d2f7761746368646f67733f7374796c653d706c6173746963)
![PyPI - License](https://pypi-camo.freetls.fastly.net/2442876dbaa56faf281ffbaa5886dcb9bc8f719d/68747470733a2f2f696d672e736869656c64732e696f2f707970692f6c2f7761746368646f67733f7374796c653d706c6173746963)
Keep an eye on the world's change.
Such as steam price fluctuation
/ new post articles
/ news on the web portal
/ server api health
/ binge-watching
, and so on...
Intro
- This is a web app based on fastapi, databases, uniparser, torequests.
- Smoothly deploy it by pip:
pip install -U watchdogs;python3 -m watchdogs
- Simple to create new crawler with the Web UI, not like old ways to duplicate crawler code.
- All the crawlers keep runing in the async environment.
- Almost all the elements have a title attribute to describe the features in the Web UI, which means docs lay on the UI.
- Release your hands from repetitive refreshing on the browser.
- Subscribe the change events with RSS reminder extensions, such as Feedbro or RSS Feed Reader.
- Or implement custom reminders by inheriting
watchdogs.callbacks.Callback
.
Usage
-
pip install -U watchdogs
-
python -m watchdogs
-
Open with a browser: http://127.0.0.1:9901
Command line args
python -m watchdogs -- -h
- db_url:
sqlite / mysql / postgresql(not test) url, which databases supports. Defaults to 'sqlite:///{HOME_PATH}/watchdogs/storage.sqlite'
- password:
init password, if null can be set on the first visit on web.
- mute_std_log:
remove stdout log for clean stream
- mute_file_log:
ignore file log located at {HOME_PATH}/watchdogs folder.
- md5_salt:
md5_salt for custom md5(password) / md5(rss_tag)
- config_dir:
config dir to save the logs and config files, if using sqlite include sqlite file. defaults to {HOME_PATH}/watchdogs
- use_default_cdn:
If Config.cdn_urls not set, and use_default_cdn is True, will use online js/css cdn from staticfile.org.
- **uvicorn_kwargs:
uvicorn startup kwargs, such as port, host. Which can be set like:
python -m watchdogs --port=9999 --host=127.0.0.1
Quick Start to Create New Task
Screenshots
- Welcome Page (Tasks Page).
Here you can see all the tasks meta, goto RSS / Mobile Lite Page, and do some operations to the tasks.
- New Task Page.
Here based on the latest uniparser web app, to create new rules and also tasks.
- Rules Page.
Do some operations for the rules.
- API page.
Based on fastapi
/docs
which is generated automatically.
- Mobile Page (Lite View).
For mobile phone to glimpse the latest result for the current 30 tasks.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for watchdogs-1.5.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d7b00ace1230717f9b68a7c1c094a4a633fc0305392a510a8ad1b98ef82e4f3 |
|
MD5 | b0a1647f7fec8acbe003a774cdf00221 |
|
BLAKE2b-256 | 86d49a75dc046b501e7db8e9865cd5eca1d7ddd388d49411891890fc64c37e99 |