eds4jinja2

Embed the data-source specification directly in your Jinja2 templates. Declare how to fetch the data inside the template, and eds4jinja2 resolves it at render time — so you build dynamic, data-driven reports (HTML, LaTeX/PDF, or any text format) from SPARQL endpoints, in-memory RDF graphs, and tabular/JSON files without writing data-loading glue code.

Why eds4jinja2?

Generating a report usually tangles two jobs: fetching the data (Python code that queries endpoints, reads files, shapes DataFrames and assembles a context dict) and presenting it (a template). Every change to what the report shows forces a round-trip through the Python code.

eds4jinja2 collapses that coupling: the template itself declares its data sources. Your Python code merely points the engine at a template and renders it — it stays agnostic of what data the report displays.

This pays off when you:

• iterate quickly on report content and queries — change the template, not the code;
• want queries and their visualisation to live together and evolve together;
• build many report variants over the same data;
• need the same report to run against a live SPARQL endpoint or an in-process graph with no code change.

Problems it solves

• ❌ bespoke context-building / data-plumbing code for every report → ✅ none.
• ❌ queries in Python, layout in templates, drifting apart → ✅ co-located and easy to change.
• ❌ one source per report → ✅ one template, many heterogeneous sources (endpoint + file + graph).
• ❌ "needs a running SPARQL server" → ✅ query an in-memory rdflib/oxigraph graph, server-less.
• ❌ slow, query-bound reports → ✅ opt-in parallel pre-fetch.

How it works

eds4jinja2 adds data-source builders as Jinja globals. In a template you call one, optionally set a query, and fetch. Every fetch returns a fail-safe (content, error) pair — a failure is surfaced into the template rather than crashing the render:

{% set rows, error = from_endpoint(endpoint).with_query("SELECT * WHERE { ?s ?p ?o } LIMIT 10").fetch_tabular() %}
{% set tree, error = from_file("config.json").fetch_tree() %}

fetch_tabular() returns a pandas DataFrame; fetch_tree() returns a nested dict (JSON / SPARQL-JSON). The data context is generated during rendering, so template.render() needs no pre-built context.

Installation

pip install eds4jinja2
# optional fast in-memory SPARQL engine (oxigraph):
pip install eds4jinja2[oxigraph]

Requires Python 3.12+ (tested on 3.12, 3.13 and 3.14). On Python 3.11, pin eds4jinja2<2.

Upgrading from 1.x? 2.0 needs Python 3.12+, runs on pandas 3 / numpy 2 (text columns in tabular results have the str dtype), and no longer installs xlrd, openpyxl, toml, py-singleton or beautifulsoup4 for you. Templates and the public API are otherwise unchanged; see the CHANGELOG for the full list and migration notes.

Quick start

A. Render a template through the eds4jinja2 environment

from jinja2 import DictLoader
from eds4jinja2 import build_eds_environment

env = build_eds_environment(loader=DictLoader({
    "report.txt": "{% set rows, err = from_file('data.csv').fetch_tabular() %}"
                  "Rows: {{ rows | length }}\n{{ rows.to_string() }}",
}))
print(env.get_template("report.txt").render())   # data is fetched while rendering

B. Drive a whole report folder with the mkreport CLI

mkreport --target ./report --output ./out

The target folder follows a small convention:

report/
├── config.json     # { "template": "main.html", "conf": { "default_endpoint": "...", ... } }
├── templates/      # main.html and any includes
└── static/         # css/js/images, copied into the output (tree preserved)

config.json names the entry template and carries a conf object of variables available to all templates. Set "template_flavour_syntax": "latex" for LaTeX templates.

Data sources

In-memory graph data sources

Render reports against an in-process RDF graph — no SPARQL server required:

• from_graph(graph) (alias from_memory) — query an rdflib.Graph, a pyoxigraph store, or any query(sparql) callable you already hold.
• from_rdf(sources, engine="rdflib") — load one or more RDF files/URLs into an in-memory graph once (engine "rdflib" default, or "oxigraph") and query it; tabular and tree results.

To render an existing report against an in-memory graph with the templates unchanged, inject a builder that overrides from_endpoint (which the templates already call):

import rdflib
from eds4jinja2 import InMemorySPARQLDataSource
from eds4jinja2.services.report_builder import ReportBuilder

graph = rdflib.Graph().parse("dataset.ttl")  # the consumer owns loading / manipulation
ReportBuilder(
    "report/",
    external_data_source_builders={"from_endpoint": lambda _endpoint: InMemorySPARQLDataSource(graph)},
).make_document()

Parallel report execution

For large reports whose runtime is dominated by SPARQL query latency, set parallelism in the report config.json to pre-fetch all data concurrently before rendering:

{ "template": "report.html", "conf": {}, "parallelism": 16 }

Execution is threads-only and all-or-nothing (any fetch failure aborts the report, no partial output); results are staged in a temp folder that is cleaned up afterwards. With parallelism unset or 1 the behaviour is exactly the previous sequential render. Threaded speed-up is real for remote endpoints and oxigraph in-memory graphs (both release the GIL); rdflib in-memory queries are GIL-bound (correct, limited speed-up).

Several documents from one template

Since 2.1, a ReportBuilder can render its template once per named output, each with its own template variables (for example one file per named graph):

import rdflib
from eds4jinja2.services.report_builder import ReportBuilder

builder = ReportBuilder("templates/shapes", output_path="out")

# in memory: {"shapes-g1.ttl": "...", "shapes-g2.ttl": "..."}; nothing written
texts = builder.render_documents({"shapes-g1.ttl": {"view": g1}, "shapes-g2.ttl": {"view": g2}})

# to disk, only if every text passes your check; returns the written paths
builder.make_documents(
    {"shapes-g1.ttl": {"view": g1}, "shapes-g2.ttl": {"view": g2}},
    check=lambda name, text: rdflib.Graph().parse(data=text, format="turtle"),
    autoescape=False,
)

The whole batch is validated before anything renders: names must stay inside the output folder and must not clash. Autoescaping can be set per batch, and parallelism applies as for make_document(). See the ReportBuilder section of the API reference for the full rules.

Documentation

Full documentation (concepts, getting started, every data source, the ReportBuilder API, extending with new data sources, architecture) is authored in AsciiDoc, built with Antora and published to GitHub Pages: https://meaningfy-ws.github.io/eds4jinja2/

Build and preview it locally with make preview-docs (needs Node.js).

Development

The project uses Poetry and a Makefile as the dev interface:

make install      # install with the test and lint dependency groups
make check-all    # the gate CI runs: ruff, mypy, import-linter, openspec validate, tests
make test-network # live SPARQL-endpoint tests (need network; not part of the gate)
make build        # build the sdist and wheel into dist/

Work is planned as OpenSpec changes under openspec/changes/; the merged requirements live in openspec/specs/.

Contributing

You are more than welcome to help expand and mature this project. We adhere to the Apache code of conduct; please follow it in all your interactions on the project. When contributing, please first discuss the change you wish to make via an issue, email, or another method with the maintainers before making a change.

Licence

This project is licensed under Apache License 2.0. Powered by Meaningfy.