domdown

domdown turns article-like web pages into clean, structured Markdown.

It is built for pages where the shape matters: long-form posts, research writeups, technical blogs, security reports, and other content-heavy pages that need to become readable Markdown without losing useful structure.

What it does

domdown takes care of the full HTML-to-Markdown pipeline:

• Parses messy web HTML
• Selects the main article content
• Removes navigation, promo blocks, and other chrome
• Extracts metadata
• Preserves images, tables, code blocks, links, and lists
• Optionally emits YAML frontmatter
• Renders the final Markdown document

The result is Markdown that is ready to read, reuse, archive, or feed into another model.

Why it exists

Most pages are not written like clean documents. They mix article content with menus, banners, share widgets, related links, and other page furniture.

domdown is designed for cases where you want the content to stay faithful to the original page while still producing a clean Markdown output that is easy to consume downstream.

Example

from domdown import DomdownOptions, html_to_markdown

html = """
<html>
  <head>
    <title>Credential theft campaign expands</title>
    <meta name="description" content="A concise security article." />
    <link rel="canonical" href="https://example.com/research/campaign" />
  </head>
  <body>
    <nav>Home Pricing Docs</nav>
    <article>
      <h1>Credential theft campaign expands</h1>
      <p>Researchers observed a new wave of phishing infrastructure.</p>
      <figure>
        <img src="/images/chart.png" alt="Campaign infrastructure chart" />
        <figcaption>Campaign infrastructure by week.</figcaption>
      </figure>
      <ul>
        <li>Windows targets increased.</li>
        <li>Linux staging remained stable.</li>
      </ul>
    </article>
  </body>
</html>
"""

markdown = html_to_markdown(
    html,
    DomdownOptions(
        base_url="https://example.com/research/campaign",
    ),
)

print(markdown)

Output:

---
title: Credential theft campaign expands
source: "https://example.com/research/campaign"
domdown_version: 0.3.7
description: A concise security article.
---
# Credential theft campaign expands

Researchers observed a new wave of phishing infrastructure.

![Campaign infrastructure chart](https://example.com/images/chart.png)

Campaign infrastructure by week.

- Windows targets increased.
- Linux staging remained stable.

What it preserves

domdown is optimized for article-style pages where useful structure should survive the conversion:

• Titles and headings
• Visible author and publication metadata
• Canonical URLs and source references
• Images and captions
• Tables and code blocks
• Inline links and emphasized text
• Lists, quotes, and other document structure

Using domdown

Client usage

Use html_to_markdown() when you only need the final Markdown document as a string.

from domdown import DomdownOptions, html_to_markdown

markdown = html_to_markdown(
    html,
    DomdownOptions(
        base_url="https://example.com/post",
        emit_frontmatter=False,
    ),
)

When emit_frontmatter=True or left at the default, the returned string includes YAML frontmatter followed by the Markdown body.

API usage

Use HtmlToMarkdownPipeline when you want structured output.

from domdown import DomdownOptions, HtmlToMarkdownPipeline

pipeline = HtmlToMarkdownPipeline(
    DomdownOptions(base_url="https://example.com/post")
)
result = pipeline.run(html)

print(result.document)
print(result.markdown)
print(result.cleaned_html)
print(result.frontmatter)
print(result.warnings)

HtmlToMarkdownResult exposes:

HtmlMetadata exposes:

Options

DomdownOptions controls parsing, cleanup, metadata extraction, and output shaping.

Example:

from domdown import DomdownOptions

options = DomdownOptions(
    base_url="https://example.com/article",
    frontmatter_opts={
        "title": "Example Article",
        "source": "https://example.com/article",
    },
    emit_frontmatter=True,
    preserve_images=True,
    remove_selectors=(".share-widget", ".newsletter-signup"),
)

Real-world coverage

domdown includes curated real-world HTML/Markdown pairs under tests/real/ to protect the pipeline against regressions on live site shapes.

• html/ stores the captured HTML for each case.
• raw/ stores the expected Markdown output for the same case.
• manifest.json declares the cases and their relative fixture paths.

To run the real-example suite:

pytest tests/real/test_real_examples.py -q

Public API

domdown exports these names from domdown.__init__:

from domdown import (
    DomdownOptions,
    HtmlMetadata,
    HtmlToMarkdownPipeline,
    HtmlToMarkdownResult,
    html_to_markdown,
)

Installation

Install from this repository:

pip install domdown

Install locally for development:

git clone https://github.com/juanmcristobal/domdown.git
cd domdown
pip install -e ".[dev]"

Runtime dependencies:

• beautifulsoup4
• lxml
• soupsieve
• httpx

Support & Connect

• ⭐ Star the repo if you found it useful
• ☕ Support me: Say thanks by buying me a coffee! https://buymeacoffee.com/juanmcristobal
• 💼 Open to work: https://www.linkedin.com/in/jmcristobal/

History

0.3.4 (2026-07-14)

• Improve generic article chrome cleanup for blog metadata, TOCs, CTAs, and related modules.
• Improve HTML table rendering for merged cells, uneven rows, escaped pipes, and line breaks.
• Add real-world regression fixtures for Okta and Socket security articles.

0.1.0 (2026-05-21)

• First release.

0.2.0 (2026-06-03)

• Fix release workflow checkout for PyPI publish.
• Change installation instructions to use pip install domdown (breaking change).