canonicalwebteam.store-llm
Flask extension making a Canonical store (snapcraft.io, charmhub.io, ...) readable by LLMs:
- every public page is served as Markdown at
/page.md /llms.txtlists the public pages, discovered from the routing table (llmstxt.org)/llms-full.txtconcatenates their Markdown<link rel="alternate" type="text/markdown">can be advertised from the layout
The HTML to Markdown conversion is done by canonicalwebteam.markdown-response; this package adds the store-specific wiring around it.
Installation
pip install canonicalwebteam.store-llm
Usage
from canonicalwebteam.store_llm import StoreLLM
StoreLLM(
app,
base_url="https://snapcraft.io",
site_name="Snapcraft",
summary="Snapcraft is the home of the Snap Store, where ...",
sections={
"": "Main pages",
"about": "Publishing a snap",
"docs": "Documentation",
"store": "Store",
},
section_order=["Main pages", "Publishing a snap", "Documentation", "Store"],
extra_links=[
{
"section": "Documentation",
"url": "https://snapcraft.io/docs/",
"title": "Snap documentation",
"description": "Reference and explanation for snaps and snapd.",
},
{
"section": "Optional",
"url": "https://snapcraft.io/llms-full.txt",
"title": "Every page in one file",
"description": "The pages above concatenated as Markdown.",
},
],
)
The application factory pattern works too: store_llm = StoreLLM(base_url=..., ...) then store_llm.init_app(app).
Options
| Option | Default | |
|---|---|---|
base_url |
required | Absolute URL of the site, used for the links in llms.txt |
site_name |
required | The H1 of llms.txt; also stripped from page titles (About | Snapcraft → About) |
summary |
required | The blockquote under the H1 of llms.txt |
sections |
{"": "Main pages"} |
Section label per first path segment. "" is for top-level pages such as /about; "about" for /about/publish. Unknown prefixes go to Other pages |
section_order |
order of sections |
Section headings in the order they appear. Other pages follows, then any unlisted section alphabetically, then Optional last so agents can skip it |
extra_links |
[] |
Links discovery cannot find: {"section", "url", "title", "description"} dicts |
is_private |
is_login_gated |
callable(view) -> bool for pages with no Markdown version. The default recognises the stores' login_required decorator. None makes every page public |
strip_elements / strip_classes |
Vanilla-aware lists | Passed to markdown-response |
markdown_cache_control |
private, max-age=3600 |
Cache-Control of Markdown responses |
llms_cache_control |
public, max-age=43200, ... |
Cache-Control of /llms.txt and /llms-full.txt |
What the store needs to provide
Templates. Discovery parses templates rather than rendering them, so every public page must define meta_title and meta_description blocks with static text, directly or through a layout it extends:
{% block meta_title %}About Snaps | Snapcraft{% endblock %}
{% block meta_description %}Snaps are app packages for desktop, cloud and IoT.{% endblock %}
A page opts out of llms.txt with <meta name="robots" content="noindex">, the same signal that keeps it out of search results. Layouts and partials (file names starting with _) are never listed.
Content. Markdown is extracted from <main id="main-content"> (falling back to <body>). Add data-md-strip to interactive chrome that reads badly as text: copy buttons, share widgets, charts.
Alternate link. Templates get markdown_path, the Markdown URL path of the current page or None for private pages:
{% if markdown_path %}
<link rel="alternate" type="text/markdown" href="https://snapcraft.io{{ markdown_path }}" />
<link rel="describedby" href="https://snapcraft.io/llms.txt" />
{% endif %}
Sitemap. app.extensions["store_llm"].sitemap_paths() returns the same pages, so the sitemap and llms.txt never drift.
Build step
Rendering every page for llms-full.txt is too slow to do on request, so generate both files at build time and the routes serve them from the static folder (falling back to rendering live when the files are missing, for development):
python3 -m canonicalwebteam.store_llm generate webapp.app:create_app
The argument is module:attribute, either a Flask app or a factory returning one. Run it from the repository root after the frontend build, for example in the Dockerfile and rockcraft.yaml, and ignore static/llms.txt and static/llms-full.txt in git. The command reports pages without a meta_description and pages that could not be rendered.
Customising the output
The files are rendered from the store_llm/llms.txt and store_llm/llms-full.txt templates. A store can override either by adding a template with the same name to its own templates/ folder; both receive site_name, and llms.txt receives summary and sections ([{"section", "links": [{"title", "url", "description"}]}]).
How pages are discovered
A route is listed when it takes no URL parameters, answers GET, is not private, its view calls render_template("....html") with a literal name and does not redirect, and the template (or a layout it extends) has a non-empty meta_title. Routes rendering the same template or sharing a title are listed once, under the shortest path (/store, not /explore). The homepage is left out: the header of llms.txt already introduces the site.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file canonicalwebteam_store_llm-0.1.0.tar.gz.
File metadata
- Download URL: canonicalwebteam_store_llm-0.1.0.tar.gz
- Upload date:
- Size: 17.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
293fab652d01cd8a95d95eb743ac9c799bc60d1b7d18fb3ee792f8fae6bda3a0
|
|
| MD5 |
9b7299b71a4274f0cf11827ea1818dab
|
|
| BLAKE2b-256 |
cfb69157a55ba4f07293671d0cefa2e93db8172c84a4c361c720af886b3886cd
|
Provenance
The following attestation bundles were made for canonicalwebteam_store_llm-0.1.0.tar.gz:
Publisher:
publish.yaml on canonical/canonicalwebteam.store-llm
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
canonicalwebteam_store_llm-0.1.0.tar.gz -
Subject digest:
293fab652d01cd8a95d95eb743ac9c799bc60d1b7d18fb3ee792f8fae6bda3a0 - Sigstore transparency entry: 2886360430
- Sigstore integration time:
-
Permalink:
canonical/canonicalwebteam.store-llm@46dae83f254814e6d77977ff40eba5c054787b63 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/canonical
-
Access:
internal
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yaml@46dae83f254814e6d77977ff40eba5c054787b63 -
Trigger Event:
push
-
Statement type:
File details
Details for the file canonicalwebteam_store_llm-0.1.0-py3-none-any.whl.
File metadata
- Download URL: canonicalwebteam_store_llm-0.1.0-py3-none-any.whl
- Upload date:
- Size: 19.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
56a1d765608f44cbd266db8fce5785a8f789ec9a87767a363deda02d961622c9
|
|
| MD5 |
cfb3a3de6f61cba1f175a147f3aad52d
|
|
| BLAKE2b-256 |
215804f55ffac351c7aa0c5fea6ccf1dd2f0d4f19604adc082d5a4d1cda148cf
|
Provenance
The following attestation bundles were made for canonicalwebteam_store_llm-0.1.0-py3-none-any.whl:
Publisher:
publish.yaml on canonical/canonicalwebteam.store-llm
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
canonicalwebteam_store_llm-0.1.0-py3-none-any.whl -
Subject digest:
56a1d765608f44cbd266db8fce5785a8f789ec9a87767a363deda02d961622c9 - Sigstore transparency entry: 2886360436
- Sigstore integration time:
-
Permalink:
canonical/canonicalwebteam.store-llm@46dae83f254814e6d77977ff40eba5c054787b63 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/canonical
-
Access:
internal
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yaml@46dae83f254814e6d77977ff40eba5c054787b63 -
Trigger Event:
push
-
Statement type:
