Skip to main content

Logo fiction-dl (1.8.3)

Screenshot (Windows 10) Screenshot (Linux Mint 20)

fiction-dl is a command-line utility designed to download works of (fan)fiction from the web and save them in a few portable file formats. It requires a Python interpreter installed, version 3.8 or higher.

The application currently supports almost two dozen websites and six output file formats; the latter being: HTML, ODT, PDF, EPUB and MOBI. It's able to download images found in story content, as well as capable of applying basic typographic corrections to the content itself.

✿ Table of Contents

  1. Features.
  2. Installation.
  3. Usage.
  4. License.
  5. Technical.

✿ Features

Downloading stories

fiction-dl can download stories from a multitude of sites - in some cases, it also supports downloading all stories published by any specific user (which is called downloading stories from a channel).

In a select few sites, it allows you to log-in to the site using the command-line; this functionality allows the user to access subforums closed to guest viewers, like the Fandom category on AlternateHistory.com, or quarantined subreddits on Reddit.

# Category Site Authentication support Channel support
1 fanfiction Archive of Our Own ✓ yes ✓ yes [1]
2 fanfiction FanFiction.net ✗ no ✓ yes [2]
3 fanfiction FicWad.com ✗ no ✓ yes
4 fanfiction SpaceBattles.com ✓ yes ✗ no
5 fanfiction SufficientVelocity.com ✓ yes ✗ no
6 fanfiction AlternateHistory.com ✓ yes ✗ no
7 fanfiction QuestionableQuesting.com ✗ no ✗ no
8 fanfiction HarryPotterFanFiction.com ✗ no ✓ yes
9 fanfiction WhoFic.com ✗ no ✓ yes [3]
10 fanfiction SamAndJack.net ✗ no ✓ yes
11 fanfiction Ralst.com ✗ no ✗ no
12 original fiction Fiction Press ✗ no ✓ yes
13 original fiction Reddit ✓ yes ✗ no
14 original fiction Quotev ✗ no ✗ no
15 original fiction Wuxiaworld ✗ no ✗ no
16 erotica AdultFanFiction ✗ no ✓ yes
17 erotica Literotica ✗ no ✓ yes
18 erotica Hentai Foundry ✗ no ✓ yes
19 erotica NajlepszaErotyka.com.pl ✗ no ✗ no
20 erotica Nifty.org ✗ no ✗ no
21 erotica asstr.org/~Kristen ✗ no ✗ no
  • [1] fiction-dl is also capable of downloading whole collections and series.
  • [2] The application supports also downloading entire collections.
  • [3] The application supports downloading whole series as well.

It is also possible to process stories stored in local files. You can put the HTML-formatted story in a raw text file and use fiction-dl to reformat it in all of its output file formats.

Formatting stories

Downloaded stories are auto-formatted and saved in a few file formats:

File Format Requirements
HTML (.html) None.
EPUB (.epub) None.
MOBI (.mobi) Calibre installed on the machine.
OpenDocument (.odt) None.
Portable Document Format (.pdf) LibreOffice installed on the machine.

Embedding images

The application can download images found in story content and embed them in output files.

Typographic corrections

fiction-dl automatically applies basic typographic corrections to the content of downloaded stories. For example:

Source Text Corrected Text
Lorem ipsum... dolor. Lorem ipsum… dolor.
Lorem ipsum...dolor. Lorem ipsum… dolor.
Lorem ipsum...... dolor. Lorem ipsum… dolor.
Lorem ipsum , dolor. Lorem ipsum, dolor.
Lorem ipsum????? Dolor. Lorem ipsum? Dolor.
Lorem ipsum - dolor. Lorem ipsum — dolor.

Packing stories

If you're downloading a lot of short stories, you can pack them into a single file, using the "-pack" command line option. This way you're adding only a single item to your Kindle book list, instead of many.

✿ Installation

You can install fiction-dl using pip:

python3 -m pip install --upgrade fiction-dl

(If the package's already installed, this command will also update it.)

If you're running a Debian-derived Linux distribution, you might also need to install the following packages:

apt-get install libgl1-mesa-glx libglib2.0-0 libmupdf-dev

(fiction-dl uses OpenCV for processing downloaded images, which requires them to be installed. PyMuPDF requires libmupdf-dev when it's being built from the source.)

✿ Usage

To download a story from a URL, simply type:

f-dl URL

In order to download multiple stories, create a text file and place the URLs in it, each one in a separate line. Then type:

f-dl YourFilesName

Options

Option Result
-h (--help) shows the overview of command-line options
-a authenticates the user to supported sites using interactive mode
-c clears the cache before launching the application
-pack packs all downloaded stories inside one file (of each type)
-v enables the (more) verbose mode
-f overwrites output files (in case they already exist)
-d enables debug mode (saves some data useful for debugging)
-no-images disables downloading images found in story content
-persistent-cache preserves the cache after the application quits
-lo used to specify the path to the LibreOffice executable (soffice.exe)
-o used to specify the output directory path

Text File Extractor

Create a text file meant to contain the story. In its first lines type:

LOCAL TEXT STORY
*URL*
*The Title of the Story*
*The Author*
*The One-Line Summary*

Follow it by the story's content, wrapped in HTML tags. Then call:

f-dl FilePath.txt

You can insert chapter-breaks in the story by typing "CHAPTER BREAK LINE" in an empty line.

✿ License

GNU GPL 3. The text of the license is provided in the LICENSE file.

✿ Technical

Changelogs

All the changelogs can be found in the Docs/Changelogs.md file.

Documentation

You can generate code documentation using Doxygen; the relevant configuration file is Docs/Docs.doxygen. Generated files will appear in the Docs/Code Documentation directory.

Tests

Launching the Integration Test can be done by entering the Tests directory and executing the following code:

python3 "Integration Test.py"

Metadata

Release files for fiction-dl 1.8.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fiction-dl 1.8.3
File Size Uploaded
fiction-dl-1.8.3.tar.gz 92.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for fiction-dl 1.8.3
File Interpreter ABI Platform
fiction_dl-1.8.3-py3-none-any.whl Python 3 none any Details

Total release size: 263.3 kB

Release files / fiction-dl-1.8.3.tar.gz

Download URL fiction-dl-1.8.3.tar.gz
Size 92.9 kB
Tags Source
SHA-256 checksum
How to use checksums
8983840660189d1ee296398f936d3bead10a408e1708c1f12e204f38ef80a6d5
BLAKE2b-256 checksum
How to use checksums
aff82259e1e9a6e691b80975fb514ac9794a05d9daae5d76b68bd5fd337310bb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.6.1 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.9.0

Release files / fiction_dl-1.8.3-py3-none-any.whl

Download URL fiction_dl-1.8.3-py3-none-any.whl
Size 170.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
396302b0cc5dbdb0ffe71a762413745a2670bcd33b19e19cf379d2330f9b5579
BLAKE2b-256 checksum
How to use checksums
34fa2cc5266c453c9d499908a54fb6c0ed97a8ac9fdbff15e653b64520bbfb55
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.6.1 requests/2.24.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.9.0

Release history Release notifications | RSS feed

This release

1.8.3 This release

2 release files

1.8.2

2 release files

1.8.1

2 release files

1.8.0

2 release files

1.7.0

2 release files

1.6.3

2 release files

1.6.2

2 release files

1.6.1

2 release files

1.6.0

2 release files

1.5.0

1 release file

1.4.0

1 release file

1.3.0

1 release file

1.2.0

2 release files

1.1.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page