novelsave-sources 0.3.0

A collection of novel sources offering varying amounts of scraping capability.

📚 Novelsave Sources

A collection of novel sources offering varying amounts of scraping capability.

Request a new source by creating a new issue

🤖 Usage

API

This package exposes 4 basic functions that can be used to interact with the provided sources, both of novel and metadata variety.

• novel_source_types - Locate and return all the novel source types
• locate_novel_source - Locate and return the novel source parser for the url if it is supported
• metadata_source_types - Locate and return all the metadata source types
• locate_metadata_source - Locate and return the metadata source parser for the url if it is supported

novel_source_types

Finds all the novel source implementations defined by this package and returns their types.

rtype - List[Type[Source]]

locate_novel_source

Takes a url and tries to find a defined novel source that can parse the webpage. If found returns the source type otherwise raises a UnknownSourceException.

rtype - Type[Source]

metadata_source_types

Finds all the metadata source implementations defined by this package and returns their types.

rtype - List[Type[MetaSource]]

locate_metadata_source

Takes a url and tries to find a defined metadata source that can parse the webpage. If found returns the source type otherwise raises a UnknownSourceException.

rtype - Type[MetaSource]

Behaviour

HttpGateway

Crawler type which Source extends from takes a BaseHttpGateway as a dependency.

The default implementation has the following properties:

• Uses cloudscraper package, which detects Cloudflare's anti-bot pages.
• Disables SSL protection, as this seems to break most sites.

You may override this behaviour by implementing BaseHttpGateway interface, and providing it as a dependency when sources are instantiated.

Build

To update the sources segment in the README.md, run the following command:

python3 manage.py compile

📒 Sources

Novel

✅ Supported

Lang	Source	Search	Last Checked
en	https://1stkissnovel.love		2021-10-14
en	https://betwixtedbutterfly.com		2021-10-07
en	https://boxnovel.com		2021-09-07
en	https://chrysanthemumgarden.com/		2021-09-07
en	https://creativenovels.com		2021-09-17
en	https://dragontea.ink/		2021-10-07
en	https://dummynovels.com		2021-09-07
en	https://forums.spacebattles.com		2021-09-09
en	https://forums.sufficientvelocity.com		2021-09-09
en	https://novelfull.com		2021-09-07
en	https://novelfun.net		2021-09-07
en	https://novelgate.net		2021-09-03
en	https://novelonlinefull.com		2021-09-07
en	https://novelsite.net		2021-09-06
en	https://novelsrock.com		2021-09-04
en	https://peachpitting.com		2021-09-04
en	https://rainofsnow.com/		2021-09-04
en	https://readlightnovels.net		2021-09-06
en	https://readnovelfull.com/		2021-10-17
en	https://wuxiaworld.online		2021-09-07
en	https://wuxiaworld.site		2021-09-03
en	https://www.chickengege.org		2021-09-07
en	https://www.foxaholic.com		2021-09-03
en	https://www.lightnovelworld.com		2021-09-07
en	https://www.mtlnovel.com		2021-09-07
en	https://www.novelhall.com		2021-09-07
en	https://www.novelpassion.com		2021-09-04
en	https://www.readlightnovel.me		2021-09-07
en	https://www.royalroad.com		2021-09-06
en	https://www.scribblehub.com		2021-09-06
en	https://www.wattpad.com		2021-09-06
en	https://www.webnovel.com		2021-09-03
en	https://www.wuxiaworld.co		2021-09-04
en	https://www.wuxiaworld.com		2021-09-04

❌ Rejected

Lang	Source	Reason
en	https://www.fanfiction.net	Has cloudflare bot protection

Metadata

✅ Supported

Lang	Metadata Source	Last Checked
en	https://www.novelupdates.com	2021-09-03
en	https://www.wlnupdates.com/	2021-08-25

📝 Disclaimer

We are not affiliated, associated, authorized, endorsed by, or in any way officially connected with any of the sources mentioned above.

📜 License

Apache-2.0