md2pydantic

Extract structured data from messy Markdown into Pydantic v2 models.

Built for resilience against common LLM output quirks: triple-backtick wrappers, trailing prose, incomplete tables, malformed JSON, and more. One line of code turns chaotic Markdown into validated, typed Python objects.

Features

• One-liner API -- MDConverter(Model).parse_tables(md) gets you started in one line
• Markdown tables -- pipe-delimited tables become lists of Pydantic models
• JSON blocks -- fenced and inline JSON, with recovery for trailing commas, single quotes, unquoted keys, and truncated output
• YAML blocks -- fenced YAML code blocks (requires pyyaml)
• Auto-detect -- parse() tries code blocks first, then tables
• Yes/No bool coercion -- "Yes", "No", "Y", "N", "true", "false", "on", "off" all map to bool
• Null sentinel handling -- empty cells, "N/A", "NA", "null", "-", "—" become None for optional fields
• Table selection -- filter tables by heading or index in multi-table documents
• LLM-resilient -- handles unclosed code fences, trailing prose, extra backticks, and nested structures
• Pydantic v2 native -- leverages Pydantic's own type coercion (str to int, str to float, str to datetime, etc.)
• Lightweight -- only dependency is pydantic>=2.0.0

Installation

pip install md2pydantic

Or with uv:

uv add md2pydantic

Optional extras:

pip install md2pydantic[yaml]    # YAML block support (pyyaml)
pip install md2pydantic[pandas]  # DataFrame conversion (pandas)

Requires Python 3.10+.

Quick Start

Parse a Markdown Table

from pydantic import BaseModel
from md2pydantic import MDConverter

class Product(BaseModel):
    name: str
    price: float
    in_stock: bool

markdown = """
Here are the products currently available:

| name       | price | in_stock |
|------------|-------|----------|
| Widget     | 9.99  | Yes      |
| Gadget     | 24.50 | No       |
"""

products = MDConverter(Product).parse_tables(markdown)
# [Product(name='Widget', price=9.99, in_stock=True),
#  Product(name='Gadget', price=24.5, in_stock=False)]

Pydantic handles the str to float coercion. md2pydantic handles "Yes" / "No" to bool.

Parse a JSON Block

from pydantic import BaseModel
from md2pydantic import MDConverter

class ServerConfig(BaseModel):
    host: str
    port: int
    debug: bool

markdown = '''Sure! Here is the server configuration:

```json
{
    "host": "localhost",
    "port": 8080,
    "debug": true,
}

Let me know if you need anything else! '''

config = MDConverter(ServerConfig).parse_json(markdown)

ServerConfig(host='localhost', port=8080, debug=True)

Notice the trailing comma after `true` -- md2pydantic fixes that automatically.

### Parse a YAML Block

```python
from pydantic import BaseModel
from md2pydantic import MDConverter

class ServerConfig(BaseModel):
    host: str
    port: int
    debug: bool

markdown = '''Here is your config:

```yaml
host: api.example.com
port: 443
debug: false

'''

config = MDConverter(ServerConfig).parse_yaml(markdown)

ServerConfig(host='api.example.com', port=443, debug=False)

Requires `pyyaml`: install with `pip install md2pydantic[yaml]`.

### Auto-Detect Format

```python
from md2pydantic import MDConverter

# parse() tries JSON/YAML code blocks first, then falls back to tables
result = MDConverter(ServerConfig).parse(markdown)

Returns a single model instance for code blocks, or a list for tables and JSON arrays.

Select Tables by Heading

When a document contains multiple tables, filter by the preceding Markdown heading:

from pydantic import BaseModel
from md2pydantic import MDConverter

class User(BaseModel):
    name: str
    age: int
    active: bool

markdown = """
## Current Staff

| name  | age | active |
|-------|-----|--------|
| Alice | 30  | Yes    |

## Former Staff

| name  | age | active |
|-------|-----|--------|
| Bob   | 25  | No     |
| Eve   | 35  | No     |
"""

current = MDConverter(User).parse_tables(markdown, heading="Current Staff")
# [User(name='Alice', age=30, active=True)]

former = MDConverter(User).parse_tables(markdown, heading="Former Staff")
# [User(name='Bob', age=25, active=False), User(name='Eve', age=35, active=False)]

Heading matching is case-insensitive and supports substring matches. You can also select by index with index=0.

Handle Null Sentinels

Empty cells and common null placeholders become None for optional fields:

class Employee(BaseModel):
    name: str
    department: str
    salary: float | None = None

markdown = """
| name  | department  | salary |
|-------|-------------|--------|
| Alice | Engineering | 95000  |
| Bob   | Marketing   | N/A    |
| Carol | Sales       | -      |
"""

employees = MDConverter(Employee).parse_tables(markdown)
# employees[0].salary == 95000.0
# employees[1].salary is None  (from "N/A")
# employees[2].salary is None  (from "-")

Recognized null sentinels: "" (empty), "N/A", "NA", "null", "-", "—". Matching is case-insensitive.

Error Handling

from md2pydantic import MDConverter, ExtractionError

try:
    result = MDConverter(MyModel).parse_tables("no tables here")
except ExtractionError as e:
    print(e)            # Human-readable summary with line numbers
    print(e.errors)     # List of typed error details

ExtractionError is raised when:

• No structured data is found in the input
• Structured data is found but none of it validates against the model

Each error in .errors is either a TransformError (parsing failed) or ModelValidationError (Pydantic rejected the data), both with source location info.

Partial Results

When parsing tables with mixed valid/invalid rows, use partial=True to get both:

from md2pydantic import MDConverter, PartialResult

result = MDConverter(User).parse_tables(markdown, partial=True)
# result.data → list of valid User instances
# result.errors → list of typed errors with row locations
# result.has_errors → True if any rows failed

ExtractionError inherits from MD2PydanticError, so you can catch either.

Supported Formats

API Reference

MDConverter(model)

Create a converter bound to a Pydantic v2 BaseModel subclass.

converter = MDConverter(MyModel)

converter.parse_tables(markdown, *, index=None, heading=None) -> list[T]

Extract Markdown tables and return validated model instances (one per row).

• index -- only parse the table at this 0-based position (applied after heading filter)
• heading -- only parse tables under headings matching this substring (case-insensitive)
• Raises ExtractionError if no tables are found or no rows validate

converter.parse_json(markdown) -> T

Extract a JSON code block and return a single validated model instance. Tries each JSON block in document order, returning the first that validates.

• Raises ExtractionError if no JSON blocks are found or none validate

converter.parse_yaml(markdown) -> T

Extract a YAML code block and return a single validated model instance.

• Raises ExtractionError if no YAML blocks are found or none validate
• Requires pyyaml (pip install md2pydantic[yaml])

converter.parse(markdown) -> T | list[T]

Auto-detect format. Tries code blocks (JSON/YAML) first, then tables.

• Raises ExtractionError if no structured data is found or none validates

Exceptions

How It Works

md2pydantic follows a Seek, Clean, Validate pipeline:

1. Scanner -- Uses regex and heuristics to identify candidate blocks (JSON, YAML, Markdown tables) within the input. Handles triple-backtick enclosures, unclosed fences, and trailing prose.

2. Transformer -- Converts raw blocks into Python dictionaries. Fixes malformed JSON (trailing commas, single quotes, unquoted keys, truncated output). Converts table rows into dicts using headers as keys.

3. Validator -- Passes dictionaries to your Pydantic model. Pre-processes Yes/No booleans and null sentinels before handing off to Pydantic's native coercion engine.

Development

git clone https://github.com/FelipeMorandini/md2pydantic.git
cd md2pydantic
uv sync --extra dev

uv run pytest              # run tests
uv run ruff check .        # lint
uv run ruff format .       # format
uv run mypy src/md2pydantic  # type check

See CONTRIBUTING.md for more details.

License

MIT