udsearch 0.1.0

Search, match, and batch-edit Universal Dependencies treebanks

udsearch

Search, match, and batch-edit Universal Dependencies treebanks from the command line or as a Python library.

Pure Python, no external dependencies.

Install

pip install udsearch

CLI

# Search for tokens
udsearch "UPOS=NOUN & Case=Dat" -t Turkish-BOUN
udsearch "UPOS=NOUN|PROPN & deprel=nsubj" -f corpus.conllu

# Structural patterns (multi-node)
udsearch $'v: [UPOS=VERB]\ns: [UPOS=NOUN] -nsubj-> v' -f corpus.conllu
udsearch --pattern-file query.txt -f corpus.conllu

# Clustering (Grew-match style)
udsearch "UPOS=NOUN" -t Turkish-BOUN --cluster
udsearch "deprel=obl" -t Turkish-BOUN --cluster Case Number

# Batch rewrite (dry-run by default)
udsearch "UPOS=NOUN & lemma=yok" --set "UPOS=ADJ" --set "Polarity=Neg" -f tb.conllu
udsearch "UPOS=NOUN & lemma=yok" --set "UPOS=ADJ" -f tb.conllu --apply

# Structural rewrite (target specific nodes)
udsearch $'v: [UPOS=VERB]\ns: [] -nsubj-> v' --set "s.Case=Nom" -f tb.conllu --apply

# Treebank management
udsearch --list tr           # list Turkish treebanks on GitHub
udsearch --list-cached       # show downloaded treebanks

Pattern syntax

Single-node

UPOS=NOUN                    exact match
UPOS=NOUN|PROPN              alternatives
lemma=/^yap/                 regex
!PronType=Prs                negation
PronType                     feature exists
UPOS=NOUN & Case=Dat         conjunction

Structural (multi-node)

v: [UPOS=VERB]                             named node
s: [UPOS=PRON & Case=Nom] -nsubj-> v      dependency relation
!a: [UPOS=AUX] -aux-> v                   negated (must NOT exist)
d: [] -nsubj|obj-> v                       deprel alternatives
d: [] -/^nsubj/-> v                        deprel regex
d: [] -> v                                 any relation
s << v                                     linear precedence

Rewrite operations

Polarity=Neg                 add/set feature
UPOS=ADJ                    change column field
-Case                        remove feature
MISC.Lang=en                 set MISC field
s.Case=Nom                   target node in structural pattern

Library usage

from udsearch import parse_conllu, parse_pattern, search_treebank
from udsearch import parse_structural, match_structural

# Parse CoNLL-U
sentences = parse_conllu(open("corpus.conllu").read())

# Single-node search
pattern = parse_pattern("UPOS=VERB & Tense=Past")
for sent, tokens in search_treebank(sentences, pattern):
    print(sent.sent_id, [t.form for t in tokens])

# Structural search
sp = parse_structural("v: [UPOS=VERB]\ns: [UPOS=NOUN] -nsubj-> v")
for sent, bindings in search_structural(sentences, sp):
    for b in bindings:
        print(f"{b['v'].form} <- {b['s'].form}")

# Batch rewrite
from udsearch import apply_operations, parse_set_operations
ops = parse_set_operations(["Polarity=Neg"])
for sent, tokens in search_treebank(sentences, pattern):
    for token in tokens:
        changes = apply_operations(token, ops)

Dict-based API (for web apps)

from udsearch import match_structural_dicts, apply_operations_to_dicts

# Works with dict-based wordlines (e.g., from a database)
wordlines = [{"id_f": "1", "form": "cat", "upos": "NOUN", ...}, ...]
matches = match_structural_dicts("UPOS=NOUN", wordlines)
modified, changes = apply_operations_to_dicts(wordlines, "UPOS=NOUN", ["Case=Acc"])

License

MIT