Linked-Data Python

Python, with the Semantic Web in its syntax. IRIs, prefixed names, RDF literals, SPARQL variables and whole graphs written in Turtle's notation are expressions of the language — interpolated with arbitrary Python, transpiled to plain Python, running on rdflib.

@prefix sosa: <http://www.w3.org/ns/sosa/> .
@base <http://example.org/building/> .

def observation(sensor, value):
    return g{ f<sensor/{sensor}> a sosa:Sensor ;
                  sosa:madeObservation [ sosa:hasSimpleResult {value} ] }

The language also reads and queries, writes into a current graph, and turns rows into triples:

@prefix ex: <http://example.org/> .
@graph as kg
for @bindings in csv.DictReader(f):             # any iterable of mappings
    +{ e<http://example.org/{?id}> ex:value ?v }

for s, v in m{ ?s ex:value ?v }:                # a graph pattern, no engine
    ...
rows = s{ SELECT ?s WHERE { ?s ex:value ?v } }  # all of SPARQL, checked early
adult = e{ ?age >= 18 && BOUND(?name) }         # deferred, over bindings

.ldpy files are transpiled to plain Python by an island parser: the Python is copied verbatim — every valid Python file is a valid ldpy file, returned byte-identical — and only the RDF islands are parsed and rewritten. The transpiler is ~1 500 lines with no parsing dependency and sustains 56 000–110 000 source lines/s depending on island density.

Quick start

git clone https://github.com/linked-data-python/ldpy.git
cd linked-data-python && pip install -e .       # or: pip install -e .[lsp,debug]

ldpy program.ldpy             # run a file
ldpy                          # interactive console
ldpy-lsp                      # language server (LSP, stdio)
ldpy-debug program.ldpy       # debug via the shadow .py + debugpy

From Python: import ldpy; ldpy.install() then import yourmodule finds yourmodule.ldpy on sys.path.

Documentation

Read it at docs/ — start with the home page for an overview, then:

• Tutorials — first steps, then build a knowledge graph from tabular data.
• How-to guides — run & import, build graphs from tables, read and query, migrate from rdflib, VS Code, debugging, language server, highlighting.
• Reference — the language, one page per island family; SPARQL expressions; CLI; Python API; language map formats.
• Explanation — why, what real RDF code does (the corpus study that drove the language's second wave), designing the syntax, island parsing, emission & semantics, tooling, how this is tested.

Every ldpy and python block in the documentation is executed by the test suite, and its assertions are the test.

Tooling

• VS Code extension (vscode-ldpy): highlighting (TextMate + LSP semantic tokens), diagnostics as you type, completion/hover/definition, run and debug.
• Language server: dependency-free, LSP over stdio; delegates Python intelligence to an unmodified pylsp through the language map.
• Debugging: .ldpy code compiles in .ldpy coordinates, so pdb and debugpy work directly; ldpy.build also materialises real .py shadow files with JSON and Source Map v3 maps.
• Highlighting anywhere else: the package registers a Pygments lexer built on the language map — MkDocs, Sphinx and pygmentize colour .ldpy with no further setup.
• Benchmark harness (bench/): seeded random program generator and reproducible throughput campaigns.

Project

• Tests: python -m pytest tests/ -q — byte-identity over the CPython standard library, golden transpilation, RDF isomorphism against rdflib as an oracle, LSP end to end, executable documentation.
• Licence: MIT. Author: Maxime Lefrançois (Mines Saint-Étienne).
• The 2023 ANTLR-based release (v1, PyPI 0.0.4) is preliminary work, superseded by this island-parsing rewrite (the main branch of this repository; the 2023 code remains on the legacy gitlab.com/coswot/ldpy).

Design records

Every non-trivial choice in this repository is written down, one file per decision, in the pilotage repository. Comments and docs cite them by identifier — ldpy/024, vscode/103 — which resolves to design/.