A declarative HTML/XML extraction framework using lxml
Verified details
These details have been verified by PyPIProject links
GitHub Statistics
Maintainers
Project description
pagemodel
A declarative HTML/XML extraction framework built on lxml.
Features
- Unified
Fielddescriptor – handles text, attributes, JSON, fragments, and multiple values via a single class. CSS and XPath selectors are auto‑detected and pre‑compiled. - Type‑driven annotations – write
title: str = "h1"and the library automatically picks the right converter (str,int,float,bool,dict,list, Pydantic models). @fragmentdecorator – describe repeating blocks of HTML without leaving the page class.@keyvaluedecorator – turn a list of fragments into a dictionary with one line.- Built‑in Pydantic integration – validate JSON data and export directly to your models.
- Streaming parser (
StreamPage) for large XML files that cannot fit in memory. - Compact helpers
Q,Sel,Xfor rapid field creation with pipeline support (| float,| str.strip). - Fast – selectors are compiled once, results are cached, and lxml does the heavy lifting in C.
Installation
pip install pagemodel
# or with extra dependencies
pip install pagemodel[all]
Python >=3.10 is required.
Quick Start
from pagemodel import BasePage, fragment
html = """<html><body>
<h1>Catalog</h1>
<div class="product">
<span class="name">Pen</span>
<span class="price" data-value="1.5"></span>
</div>
<div class="product">
<span class="name">Paper</span>
<span class="price" data-value="2.0"></span>
</div>
</body></html>"""
class Page(BasePage):
title: str = "h1"
@fragment(".product", multiple=True)
class Product:
name: str = ".name"
price: float = ".price @data-value"
page = Page(html)
print(page.title) # Catalog
for prod in page.Product:
print(prod.name, prod.price)
print(page.export()) # full dictionary
Advanced usage
Explicit Field, JSON and Pydantic models
from pagemodel import BasePage, Field
from pydantic import BaseModel
class ProductModel(BaseModel):
name: str
price: float
class Page(BasePage):
product: ProductModel = "script[type='application/json']"
page = Page('<script type="application/json">{"name":"Pen","price":1.5}</script>')
print(page.product.name) # Pen
print(page.product.price) # 1.5
Key‑value aggregation with @keyvalue
from pagemodel import BasePage, fragment, keyvalue
@keyvalue("Spec", key_field="label", value_field="val", export_name="specs")
class Page(BasePage):
@fragment(".row", multiple=True)
class Spec:
label: str = ".key"
val: str = ".val"
page = Page("""<div class="row"><span class="key">Color</span><span class="val">Red</span></div>""")
print(page.export()) # {'specs': {'Color': 'Red'}}
Streaming large XML
from pagemodel import StreamPage, HtmlFragment, Field
class Item(HtmlFragment):
name = Field("name")
xml = """<root><item><name>Alice</name></item><item><name>Bob</name></item></root>"""
stream = StreamPage(xml, tag_callback_map={"item": Item})
for entry in stream.iter_items("item"):
print(entry) # {'name': 'Alice'} {'name': 'Bob'}
Documentation
Running tests
uv run pytest
License
This project is licensed under the MIT License – see the LICENSE file for details.
Project details
Verified details
These details have been verified by PyPIProject links
GitHub Statistics
Maintainers
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pagemodel-0.2.0.tar.gz
(29.1 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
pagemodel-0.2.0-py3-none-any.whl
(21.9 kB
view details)
File details
Details for the file pagemodel-0.2.0.tar.gz.
File metadata
- Download URL: pagemodel-0.2.0.tar.gz
- Upload date:
- Size: 29.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
85cc745a31141d7f7f2952efd6afe19627363c566c9d4af2e6eb4c87a3545adb
|
|
| MD5 |
606ef9a5ceaac374c105c151ae097e01
|
|
| BLAKE2b-256 |
040ed5210f2ab3d58600099eee8b9b773e4f5c335782d9d74ff096b035441f0e
|
Provenance
The following attestation bundles were made for pagemodel-0.2.0.tar.gz:
Publisher:
publish.yml on avalak/pagemodel
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
pagemodel-0.2.0.tar.gz -
Subject digest:
85cc745a31141d7f7f2952efd6afe19627363c566c9d4af2e6eb4c87a3545adb - Sigstore transparency entry: 1784060685
- Sigstore integration time:
-
Permalink:
avalak/pagemodel@4a5494470cedfad0a2a11f5686f00a31b6cca985 -
Branch / Tag:
refs/tags/v0.2.0 - Owner: https://github.com/avalak
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@4a5494470cedfad0a2a11f5686f00a31b6cca985 -
Trigger Event:
push
-
Statement type:
File details
Details for the file pagemodel-0.2.0-py3-none-any.whl.
File metadata
- Download URL: pagemodel-0.2.0-py3-none-any.whl
- Upload date:
- Size: 21.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8ef1e935abc5e0e8f8c2785e976473fca962e14112c4658666f661d5c1e27bc0
|
|
| MD5 |
825cba461ebdba605ab3aca53ab16856
|
|
| BLAKE2b-256 |
d120dd0eb52a74bbba959ffb6f3bffde7a359434a43eaf5bd2528f8b5ea5f526
|
Provenance
The following attestation bundles were made for pagemodel-0.2.0-py3-none-any.whl:
Publisher:
publish.yml on avalak/pagemodel
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
pagemodel-0.2.0-py3-none-any.whl -
Subject digest:
8ef1e935abc5e0e8f8c2785e976473fca962e14112c4658666f661d5c1e27bc0 - Sigstore transparency entry: 1784060891
- Sigstore integration time:
-
Permalink:
avalak/pagemodel@4a5494470cedfad0a2a11f5686f00a31b6cca985 -
Branch / Tag:
refs/tags/v0.2.0 - Owner: https://github.com/avalak
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@4a5494470cedfad0a2a11f5686f00a31b6cca985 -
Trigger Event:
push
-
Statement type:
