Convert web pages to markdown
Project description
Site to Markdown Converter
A Python package that converts web pages to clean markdown format.
Installation
pip install -e .
Or install dependencies directly:
pip install -r requirements.txt
Usage
from site2markdown import UrlToMarkdown
# Create converter instance
converter = UrlToMarkdown()
# Convert a URL to markdown
markdown = converter.convert(
url="https://www.example.com",
inline_title=True, # Include title as H1 heading
ignore_links=False, # Keep links in output
improve_readability=True # Apply readability improvements
)
print(markdown)
# Convert HTML directly
html_content = "<html><body><h1>Title</h1><p>Content</p></body></html>"
markdown = converter.convert_html(
html=html_content,
url="https://www.example.com", # Base URL for relative links
inline_title=True,
ignore_links=False,
improve_readability=True
)
print(markdown)
Features
- Converts web pages to clean markdown format
- Supports special handling for:
- Stack Overflow questions and answers
- Apple Developer documentation
- Wikipedia articles
- Medium articles
- Preserves code blocks and tables
- Removes unwanted elements (scripts, styles)
- Applies domain-specific filters
- Makes relative URLs absolute
- Optional readability improvements using Mozilla's Readability algorithm
Options
inline_title(bool): Include page title as H1 heading (default: True)ignore_links(bool): Remove all links from output (default: False)improve_readability(bool): Apply readability algorithm to extract main content (default: True)
License
MIT License
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
site2markdown-1.0.0.tar.gz
(11.0 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file site2markdown-1.0.0.tar.gz.
File metadata
- Download URL: site2markdown-1.0.0.tar.gz
- Upload date:
- Size: 11.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a706e8724c52f32cbb6f5ac890ffe505946ef5f618f68a65708d01d85f7e53f7
|
|
| MD5 |
a7190db2aebb77f6ef495c64e44e70f8
|
|
| BLAKE2b-256 |
3e95df1ca66997bd895bba88e3f06dde6dc592a08af0ca7fbf78aa5bdf768aee
|
File details
Details for the file site2markdown-1.0.0-py3-none-any.whl.
File metadata
- Download URL: site2markdown-1.0.0-py3-none-any.whl
- Upload date:
- Size: 12.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
58f8325d310b992aeb2195e0c1b35696d3e9b608e99ba2535c9f577bc7bd7eef
|
|
| MD5 |
3d4453bd501843f836abea1c8da8f67c
|
|
| BLAKE2b-256 |
1d6cc0cacbd6214c505ff96eb22838909c0367b2a077181f6b89f8ffbef32cc7
|
