Skip to main content

xsget

Console tools to download online novel and convert to text file.

Installation

Stable version From PyPI using uv:

uv tool install xsget
uv tool install playwright
playwright install

Upgrade to latest stable version:

uv tool upgrade xsget

Latest development version from GitHub:

python3 -m pip install -e git+https://github.com/kianmeng/xsget.git
playwright install

xsget

xsget -h
usage: xsget [-q] [-e] [-d] [-h] [-V] [-l CSS_PATH] [-P URL_PARAM]
             [-g [FILENAME] | -c [FILENAME]] [-r] [-t] [-b] [-bs SESSION]
             [-bd DELAY] [--tor] [-cc CONCURRENCY] [--retries RETRIES]
             [-od OUTPUT_DIR]
             [URL]

xsget is a console app that crawl and download online novel.

website: https://github.com/kianmeng/xsget
changelog: https://github.com/kianmeng/xsget/blob/master/CHANGELOG.md
issues: https://github.com/kianmeng/xsget/issues

positional arguments:
  URL   set url of the index page to crawl

options:
  -q, --quiet
        suppress all logging
  -e, --env
        print environment information for bug reporting
  -d, --debug
        show debugging log and stacktrace
  -h, --help
        show this help message and exit
  -V, --version
        show program's version number and exit
  -l CSS_PATH, --link-css-path CSS_PATH
        set css path of the link to a chapter (default: 'a')
  -P URL_PARAM, --url-param-as-filename URL_PARAM
        use url param key as filename (default: '')
  -g [FILENAME], --generate-config-file [FILENAME]
        generate config file from options (default: 'xsget.toml')
  -c [FILENAME], --config-file [FILENAME]
        load config from file (default: 'xsget.toml')
  -r, --refresh
        refresh the index page
  -t, --test
        show extracted urls without crawling
  -b, --browser
        crawl by actual browser (default: 'False')
  -bs SESSION, --browser-session SESSION
        set the number of browser session (default: 5)
  -bd DELAY, --browser-delay DELAY
        set the millisecond to wait for page to load in browser (default: 0)
  --tor
        routes http requests through Tor SOCKS5 proxy at socks5://127.0.0.1:9150 (default: 'False')
  -cc CONCURRENCY, --concurrency CONCURRENCY
        set the maximum number of concurrent downloads (default: 5)
  --retries RETRIES
        set the number of retries for transient download failures (default: 3)
  -od OUTPUT_DIR, --output-dir OUTPUT_DIR
        set default output folder (default: 'output')

examples:
  xsget http://localhost
  xsget http://localhost/page[1-100].html
  xsget -g -l "a" -p "id" http://localhost

xstxt

xstxt -h
usage: xstxt [-q] [-e] [-d] [-h] [-V] [-pt CSS_PATH] [-pb CSS_PATH]
             [-la LANGUAGE] [-ps SEPARATOR] [-rh REGEX REGEX]
             [-rt REGEX REGEX] [-bt TITLE] [-ba AUTHOR] [-ic INDENT_CHARS]
             [-fw] [-oi] [-ow] [-i GLOB_PATTERN] [-x GLOB_PATTERN]
             [-l TOTAL_FILES] [-w WIDTH] [-o FILENAME] [-od OUTPUT_DIR]
             [-cc CONCURRENCY] [-y] [-p] [-g [FILENAME] | -c [FILENAME]] [-m]

xstxt is a console app that extract content from HTML to text file.

website: https://github.com/kianmeng/xsget
changelog: https://github.com/kianmeng/xsget/blob/master/CHANGELOG.md
issues: https://github.com/kianmeng/xsget/issues

options:
  -q, --quiet
        suppress all logging
  -e, --env
        print environment information for bug reporting
  -d, --debug
        show debugging log and stacktrace
  -h, --help
        show this help message and exit
  -V, --version
        show program's version number and exit
  -pt CSS_PATH, --title-css-path CSS_PATH
        set css path of chapter title (default: 'title')
  -pb CSS_PATH, --body-css-path CSS_PATH
        set css path of chapter body (default: 'body')
  -la LANGUAGE, --language LANGUAGE
        language of the ebook (default: 'zh')
  -ps SEPARATOR, --paragraph-separator SEPARATOR
        set paragraph separator (default: '\n\n')
  -rh REGEX REGEX, --html-replace REGEX REGEX
        set regex to replace word or phrase in html file
  -rt REGEX REGEX, --txt-replace REGEX REGEX
        set regex to replace word or phrase in txt file
  -bt TITLE, --book-title TITLE
        set title of the novel (default: '不详')
  -ba AUTHOR, --book-author AUTHOR
        set author of the novel (default: '不详')
  -ic INDENT_CHARS, --indent-chars INDENT_CHARS
        set indent characters for a paragraph (default: '')
  -fw, --fullwidth
        convert ASCII character from halfwidth to fullwidth (default: 'False')
  -oi, --output-individual-file
        convert each html file into own txt file
  -ow, --overwrite
        overwrite output file
  -i GLOB_PATTERN, --input GLOB_PATTERN
        set glob pattern of html files to process (default: '['./*.html']')
  -x GLOB_PATTERN, --exclude GLOB_PATTERN
        set glob pattern of html files to exclude (default: '[]')
  -l TOTAL_FILES, --limit TOTAL_FILES
        set number of html files to process (default: '3')
  -w WIDTH, --width WIDTH
        set the line width for wrapping (default: 0, 0 to disable)
  -o FILENAME, --output FILENAME
        set output txt file name (default: 'book.txt')
  -od OUTPUT_DIR, --output-dir OUTPUT_DIR
        set output directory (default: 'output')
  -cc CONCURRENCY, --concurrency CONCURRENCY
        set the maximum number of concurrent file processing (default: 5)
  -y, --yes
        yes to prompt
  -p, --purge
        remove extracted files specified by --output-dir option (default: 'False')
  -g [FILENAME], --generate-config-file [FILENAME]
        generate config file from options (default: 'xstxt.toml')
  -c [FILENAME], --config-file [FILENAME]
        load config from file (default: 'xstxt.toml')
  -m, --monitor
        monitor config file changes and re-run when needed

examples:
  xsget -g
  xstxt --input *.html
  xstxt --output-individual-file --input *.html
  xstxt --config --monitor

Copyright (C) 2021,2022,2023,2024,2025,2026 Kian-Meng Ang

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Metadata

Release files for xsget 0.1.117

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xsget 0.1.117
File Size Uploaded Provenance Actions
xsget-0.1.117.tar.gz 64.0 kB Without provenance Details for xsget-0.1.117.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for xsget 0.1.117
File Interpreter ABI Platform Actions
xsget-0.1.117-py3-none-any.whl Python 3 none any Details for xsget-0.1.117-py3-none-any.whl

Total release size: 110.0 kB

Release files / xsget-0.1.117.tar.gz

Download URL xsget-0.1.117.tar.gz
Size 64.0 kB
Tags Source
SHA-256 checksum
How to use checksums
4295dfc46ac7ea35615d045f9c1c39806ce8039756a4590e2a95417bf07f08e9
BLAKE2b-256 checksum
How to use checksums
3cf7c9724b56817a69e4e219c39f2a6504c13d2af54cf730abb851f7bcdc505d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.10","id":"stonking","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / xsget-0.1.117-py3-none-any.whl

Download URL xsget-0.1.117-py3-none-any.whl
Size 46.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
542a10ca38f6192104aeea734800ef794e64a6271c2e439f53353bb432e37ae0
BLAKE2b-256 checksum
How to use checksums
e687556150c7ca14332998c7c1ab8f8cc4552858654656849abda0c651df1ce7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.10","id":"stonking","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for xsget 0.1.117

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.117 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page