Skip to main content

novel grab crawler module using python3 and lxml

multiprocesssing with multithread version

winxos, AISTLAB Since 2017-02-19

INSTALL:

pip3 install aistlab_novel_grab

1. USAGE:

RUN COMMAND IN CONSOLE:

novel_grab http://the_url_of_novel_chapters_page

EXAMPLE:

novel_grab http://book.zongheng.com/showchapter/654086.html

SUPPORTED SITES: * http://book.zongheng.com * http://www.aoyuge.com * http://www.quanshu.net

2. USAGE AS PYTHON MODULE:

    from novel_grab.novel_grab import Downloader
    d = Downloader()
    print(d.get_info())
    if d.set_url('http://book.zongheng.com/showchapter/221579.html'):
        d.start()

**TIPS** \* When d = Downloader(), d.get\_info() can get supported
sites info. \* Once d.set\_url(url) will return the url is valid or
not. \* Of course you can use d.get\_info() to access the state of d
at any time. \* While finished, will create :math:`novel_name`.zip
file in your current path, default zip method using
zipfile.ZIP\_DEFLATED

Just for educational purpose, take care of yourself.

Release files for AISTLAB_novel_grab 1.2.12

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for AISTLAB_novel_grab 1.2.12
File Size Uploaded
AISTLAB_novel_grab-1.2.12.tar.gz 5.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for AISTLAB_novel_grab 1.2.12
File Interpreter ABI Platform
AISTLAB_novel_grab-1.2.12-py2.py3-none-any.whl Python 2, Python 3 none any Details

Total release size: 15.0 kB

Release files / AISTLAB_novel_grab-1.2.12.tar.gz

Download URL AISTLAB_novel_grab-1.2.12.tar.gz
Size 5.9 kB
Tags Source
SHA-256 checksum
How to use checksums
369b6530d185ca642ec5ec3824f104e4222c8a2738dc86419cf6529b27a456f2
BLAKE2b-256 checksum
How to use checksums
6c516ea42eb72b6c75b0e4155469063b02c9d606775672a4f9ab457ef5a262f9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / AISTLAB_novel_grab-1.2.12-py2.py3-none-any.whl

Download URL AISTLAB_novel_grab-1.2.12-py2.py3-none-any.whl
Size 9.1 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
9945cbf8801f0bef2a8004e0b830ffbaea48702e51b30dc93113cdcfdfb93b4d
BLAKE2b-256 checksum
How to use checksums
76ac2cf2ee7f72ac0a5a09c25924bc2fa58ca02f37f1f1ad1bbe198270cfb717
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release history Release notifications | RSS feed

This release

1.2.12 This release

2 release files

1.2.10

2 release files

1.2.8

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page