Overview
The docscraper package is a scrapy spider for crawling a given set of
websites and dowloading all available documents with a given set of file
extensions. The package is intended to be called from a Python script.
Getting Started
You can get started by downloading the package with pip:
$ pip install docscraper
Once the package is installed, you can use it with scrapy directly in your Python script to download files from websites as follows:
>>> import docscraper
>>> allowed_domains = ["books.toscrape.com"]
>>> start_urls = ["https://books.toscrape.com"]
>>> extensions = [".html", ".pdf", ".docx", ".doc", ".svg"]
>>> docscraper.crawl(allowed_domains, start_urls, extensions=extensions)
1 maintainer
Release files for docscraper 2.0.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| docscraper-2.0.7.tar.gz | 10.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| docscraper-2.0.7-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 22.6 kB
Release files / docscraper-2.0.7.tar.gz
| Download URL | docscraper-2.0.7.tar.gz |
|---|---|
| Size | 10.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
29a5362ffcc939366e2dd0380795ebd78f887be378421eabd5447d19f7d9cdf3
|
|
BLAKE2b-256 checksum How to use checksums |
bae4d0536d5a703316207221e815cc7382c91f5dee7c585d6c8236b1855ce566
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/52.0.0.post20210125 requests-toolbelt/0.9.1 tqdm/4.48.0 CPython/3.8.8
|
Release files / docscraper-2.0.7-py3-none-any.whl
| Download URL | docscraper-2.0.7-py3-none-any.whl |
|---|---|
| Size | 12.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
25cbe46842e75cc6576cbc305b418688a60f5bb5c79a221055c2b8f6d1293d1d
|
|
BLAKE2b-256 checksum How to use checksums |
8af418140f3ec189a01bbd5337d71223a4976039104a3af070f8cde1073a3ee0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/52.0.0.post20210125 requests-toolbelt/0.9.1 tqdm/4.48.0 CPython/3.8.8
|
