Sitemap generator library for python. Fork from https://github.com/Haikson/sitemap-generator.
Installing
pip install nix-sitemap-generator
Usage
1. Import crawler from pysitemap
from pysitemap import crawler
2. Call crawler()
crawler(
'https//site.com', out_file='debug/sitemap.xml', exclude_urls=[".pdf", ".jpg", ".zip"],
http_request_options={"ssl": False}, parser=Parser
)
Example
import sys
import logging
from pysitemap import crawler
from pysitemap.parsers.lxml_parser import Parser
if __name__ == '__main__':
if '--iocp' in sys.argv:
from asyncio import events, windows_events
sys.argv.remove('--iocp')
logging.info('using iocp')
el = windows_events.ProactorEventLoop()
events.set_event_loop(el)
# root_url = sys.argv[1]
root_url = 'https://www.haikson.com'
crawler(
root_url, out_file='debug/sitemap.xml', exclude_urls=[".pdf", ".jpg", ".zip"],
http_request_options={"ssl": False}, parser=Parser
)
Changes
v. 0.10.1
Refactored the code to make it more readable.
Removed prints() calls from code.
Added verbose mode to crawler().
Added type hints to crawler() arguments.
Add ValueError handling when try to add_signal_handler()
1 maintainer
Release files for nix-sitemap-generator 0.10.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| nix-sitemap-generator-0.10.1.tar.gz | 13.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| nix_sitemap_generator-0.10.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 29.2 kB
Release files / nix-sitemap-generator-0.10.1.tar.gz
| Download URL | nix-sitemap-generator-0.10.1.tar.gz |
|---|---|
| Size | 13.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5c448ead092a83ae14c61cfb95b1dfe3f8b627e8e8401a01e69750f91f3fab9b
|
|
BLAKE2b-256 checksum How to use checksums |
a9509a4861b00146dfb984a6bed78c7e5bbc07b127aa2b554a38db6f11d21889
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.11.8
|
Release files / nix_sitemap_generator-0.10.1-py3-none-any.whl
| Download URL | nix_sitemap_generator-0.10.1-py3-none-any.whl |
|---|---|
| Size | 16.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0f28d09d7aafa0e61a23846a1ab5e01b37fe894c442209e99c3f80a56e75ba75
|
|
BLAKE2b-256 checksum How to use checksums |
03c7762813012d878fb9a59dd0918578327df55eea004034a7b102e12586f9a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.11.8
|
