vyyhti

Tangle (Finnish: vyyhti) -- scan and process embedded processing instructions in text documents.

Requires Python 3.11 or later. YAML support uses ruamel.yaml (default) or PyYAML (pip install vyyhti[pyyaml]).

Install

pip install vyyhti

Quickstart

Run vyyhti scan with a text file to list every detected embedding:

$ vyyhti scan doc.md
line_command:1:0: '\\newpage'
inline_code:9:22: '`$.document.notes[*].group_ids[*]`'
line_command:11:0: '\\columns=10%,,%30%'
block_pi:21:0: '```{.text} <!--json-path-list-->'

For a dedicated feature walkthrough you can follow in minutes visit quickstart. A step-by-step build of a JSONPath expression linter for Markdown documents is provided in the tutorial.

Embedding kinds

Inline code

A single-backtick span on any non-fenced line:

The expression `$.notes[*].id` must yield at least one result.

Line command

A line whose only content is a backslash command (as used by liitos and LaTeX preprocessors):

\newpage
\columns=10%,,30%

Block processing instruction

A fenced code block whose info string contains an HTML processing instruction comment:

```{.text} <!--json-path-list-->
$.document.notes[*].group_ids[*]
$.vulnerabilities[*].flags[*].group_ids[*]
```

The PI name (json-path-list above) is extracted and available to handlers. The fence body is preserved verbatim.

Library API

scan(text, config=None)

Scans a text string and returns a list of Location objects -- one per detected embedding.

from vyyhti import scan, LocationKind

locs = scan(open('doc.md').read())
for loc in locs:
    print(loc.kind, loc.line, loc.text)

Each Location is a frozen dataclass:

Field	Type	Meaning
kind	LocationKind	INLINE_CODE, LINE_COMMAND, or BLOCK_PI
line	int	1-based line number of the embedding start
col	int	1-based column for INLINE_CODE; 0 for whole-line kinds
text	str	Raw matched text (backtick span / command / opening fence line)
body	str \ None	For BLOCK_PI: fence body; None otherwise
pi	str \ None	For BLOCK_PI, comment syntax: content of <!--...-->; None otherwise
attrs	dict[str,str] \ None	For BLOCK_PI, attribute syntax: {.cls key="val" ...}; None otherwise -- a BLOCK_PI carries exactly one of pi/attrs, never both

text never includes a line terminator, for a different reason per kind: INLINE_CODE and LINE_COMMAND never matched one to begin with (a backtick span can't itself contain a newline; LINE_COMMAND's own captured group excludes it); BLOCK_PI's text happens to coincide with a whole source line, and scan()'s own line splitting is what strips it there -- the one case where a consumer could plausibly expect a terminator and not get one. Handler.rewrite (below) exists so you never have to reason about this directly.

run(text, handlers, stages=None, config=None)

Scans the text and applies a list of handlers through the processing pipeline. Returns a RunResult -- a NamedTuple of (embeddings, findings, text), so it still unpacks positionally (embeddings, findings, text = run(...)) while also supporting .embeddings/.findings/.text by name. text is always a real string: the original input back unchanged if no handler's rewrite() returned a replacement, or the reconstructed document if one or more did.

from vyyhti import run, Handler, LocationKind

class MyHandler(Handler):
    name = 'my-handler'

    def identify(self, location):
        return location.kind == LocationKind.INLINE_CODE

    def parse(self, location):
        return location.text[1:-1]   # strip backticks

    def verify(self, location, payload):
        return [] if payload.startswith('$') else ['not a JSONPath']

embeddings, findings, text = run(text, [MyHandler()])

Each matched Location becomes an Embedding (with handler name, payload, and replacement set). Each problem becomes a Finding (with stage, message, and level).

Pass stages={'parse', 'verify'} to limit which pipeline stages execute. Pass a ScannerConfig to configure which embedding kinds are scanned.

Rewriting matched text with Handler.rewrite

A handler that needs to upgrade or normalize matched syntax -- not just report on it -- overrides rewrite(location, payload) -> str | None, returning the location's own replacement line content (without a trailing line terminator; run() reattaches whatever the original line actually had), or None for no change:

class UpgradeLegacyPi(Handler):
    name = 'upgrade-legacy-pi'

    def identify(self, location):
        return location.kind == LocationKind.BLOCK_PI and location.pi is not None

    def rewrite(self, location, payload):
        return location.text.replace(f'<!--{location.pi}-->', '').rstrip()

_, _, upgraded_text = run(text, [UpgradeLegacyPi()])

Only BLOCK_PI and LINE_COMMAND support rewrite() -- both are whole-line matches by construction, so replacing "the line this match is on" is unambiguous. INLINE_CODE is a sub-line span that may share its line with unrelated content; run() raises ValueError if a handler returns a replacement for one, rather than silently clobbering that content. A rewrite() that raises has its exception converted to a Finding (stage='rewrite') instead, same as parse/process -- the attempted replacement is not applied.

ScannerConfig

Controls which embedding kinds are active and overrides the default match patterns. Pass an instance to scan() or run().

from vyyhti import scan, ScannerConfig

cfg = ScannerConfig(line_command=False)   # skip \commands
locs = scan(text, cfg)

Field	Type	Default	Meaning
inline_code	bool	True	Scan for inline-code spans
line_command	bool	True	Scan for backslash line commands
block_pi	bool	True	Scan for fenced-block PIs
block_pi_pi_pattern	str	<!--(.*?)-->	Regex for PI comment in fence info string
line_command_pattern	str	^\s*(\\...)$	Regex for line commands

Load from a YAML file (kebab-case keys) with load_scanner_config(path) from vyyhti._config.

Handler

An abstract base class. Subclass it and implement the methods you need.

Method	Signature	Required	Default
name	str class attribute	yes	--
identify	(location) -> bool	yes	--
parse	(location) -> Any	no	returns location.text
verify	(location, payload) -> list[str]	no	returns []
validate	(location, payload) -> list[str]	no	returns []
process	(location, payload) -> Any	no	returns None
rewrite	(location, payload) -> str \ None	no	returns None

Finding

Field	Type	Meaning
location	Location	The embedding where the problem was found
stage	str	'parse', 'verify', 'validate', 'process', 'rewrite'
message	str	Human-readable description
level	str	'error' (default), 'warning', or 'info'

Design

Handlers live in the tools that use vyyhti, not in the library itself. scan() finds all structural embedding locations; each handler decides what it claims via identify(). The pipeline then applies only the stages the handler cares about. Handlers for different embedding kinds can coexist in the same run() call.

Exit codes

• 0 -- success.
• 1 -- error: file not found, unreadable, or missing required argument.

See also

man vyyhti

Design and requirements

Document	Identifier	File
Software Requirements Specification	VYY-SRS-001	requirements/srs/
Software Design Description	VYY-SDD-001	design/sdd/

Both documents follow the MIL-STD-498 DID structure and are rendered into the documentation site alongside the quickstart and tutorial.

Changes

See docs/changes.md for the release history.

Coverage

The test suite maintains 99% branch coverage. The HTML report (if generated) is in site/coverage/.

SBOM

Runtime dependency information is published in docs/sbom/ in SPDX 3.0 (JSON-LD) and CycloneDX 1.6 (JSON) formats. See docs/sbom/README.md for the component inventory and validation guide.