Clip web pages html and store in Django
Project description
Django KAOS web clipper
Clip web pages html and store in Django
Quick start
pip install dj-kaos-webclipper
Add webclipper
to INSTALLED_APPS
if you want to use the generic webclip model, or extend AbstractWebClip
if you
want to customize it on your application.
If you go with the generic WebClip model, you can get REST endpoint by using the router defined in rest.routes
.
Your end users will send the title, URL and HTML content of the page using the REST API endpoint, and they get saved to
the models. You can extend the WebClip model in a proxy model, and extend RawItemMixin
from the library, which gives
you an interface to parse the page using Scrapy's API.
Development and Testing
IDE Setup
Add the example
directory to the PYTHONPATH
in your IDE to avoid seeing import warnings in the tests
modules. If
you are using PyCharm, this is already set up.
Running the Tests
Install requirements
pip install -r requirements.txt
For local environment
pytest
For all supported environments
tox
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dj-kaos-webclipper-0.1.0rc1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a0884d4d0ebe4b0bf806d1b5351b24eebe362df1d598665ce9de3c5adf0f50d8 |
|
MD5 | 9508e1b43e5a4ce259d297611c1a6a58 |
|
BLAKE2b-256 | 086e2e7adfc47ba146080188d501cef696c6d289f89dbf3eded4610dce29f951 |
Hashes for dj_kaos_webclipper-0.1.0rc1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5a35d7bf314cb7445c18790518447e29af30d931a5f574da1bc39f1d9bf1b5b1 |
|
MD5 | 6cd90fe6f207d7abdd2dacc6ba4ac1f4 |
|
BLAKE2b-256 | 14b06c063f2c6c215e86a4be9d619504afd88ea3e3ac320e211d541e26fd2ded |