Skip to main content

ryokai 了解

Ryokai (了解, "understood / got it") — a unified Python library for semantic machine-translation evaluation, combining the strengths of MEANT 2.0, XMEANT, YiSi-1/2, WOLVESAAR, and SimAlign behind one clean API on top of modern multilingual embeddings.

Pure PyTorch + HuggingFace transformers — no Stanza, no spaCy, no external parsers. Two HF models cover all 13 supported languages (en, de, fr, es, cs, fi, hi, lv, pl, ro, ru, tr, zh) in a single install:

Both are one-line swappable for any modern multilingual encoder (Qwen3-Embedding, Jina v3, BGE-M3, Nemotron-8B…) — see Embedding backbones in DOCUMENTATION.md.

Install

pip install ryokai

Quickstart

from ryokai import Ryokai

scorer = Ryokai()
src_lang, tgt_lang = "en", "ja"

# Most common: reference-free, word alignment + embedding
# (XMEANT-lite / YiSi-2 / Doc-embedding adequacy cross-lingual)
scorer.score(source=src, hypothesis=hyp,
             source_lang=src_lang, target_lang=tgt_lang)

Variants

One .score() call, four modes, dispatched by which arguments you pass. srl=False is the default — ryokai is no longer MEANT-first.

from ryokai import Ryokai
scorer = Ryokai()
src_lang, tgt_lang = "en", "ja"

# Reference-free, word alignment + embedding (default, most common)
# E.g. Doc-embedding adequacy / YiSi-2 / XMEANT-lite
scorer.score(source=src, hypothesis=hyp,
             source_lang=src_lang, target_lang=tgt_lang)

# Reference-based, word alignment + embedding
# E.g. Doc-embedding adequacy / WOLVESAAR / YiSi-1 / SimAlign style
scorer.score(reference=ref, hypothesis=hyp, target_lang=tgt_lang)

# Reference-free, frame-based — XMEANT proper
scorer.score(source=src, hypothesis=hyp,
             source_lang=src_lang, target_lang=tgt_lang, srl=True)

# Reference-based, frame-based — MEANT 2.0
scorer.score(reference=ref, hypothesis=hyp, target_lang=tgt_lang, srl=True)

See DOCUMENTATION.md for flags, aligner choices, embedding-backbone swaps, AER evaluation harness, CLI, architecture, and custom role weights.

References

Ryokai is glue around several published techniques — credit belongs to their authors.

Technique Year Citation Category
MEANT 2011 Lo & Wu. MEANT: An inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility based on semantic roles. ACL 2011. Semantic-frame MT evaluation
XMEANT 2014 Lo, Beloucif, Saers & Wu. XMEANT: Better semantic MT evaluation without reference translations. ACL 2014 (Short Papers). Semantic-frame MT evaluation
MEANT 2.0 2017 Lo. MEANT 2.0: Accurate semantic MT evaluation for any output language. WMT 2017. Semantic-frame MT evaluation
Doc-embedding adequacy 2015 Vela & Tan. Predicting Machine Translation Adequacy with Document Embeddings. WMT 2015. Embedding-based MT evaluation
WOLVESAAR 2016 Bechara, Gupta, Tan, Orăsan, Mitkov & van Genabith. WOLVESAAR at SemEval-2016 Task 1: Replicating the Success of Monolingual Word Alignment and Neural Embeddings for Semantic Textual Similarity. SemEval-2016. Embedding-based MT evaluation
YiSi 2019 Lo. YiSi — a Unified Semantic MT Quality Evaluation and Estimation Metric for Languages with Different Levels of Available Resources. WMT 2019. Embedding-based MT evaluation
Monolingual aligner 2014 Sultan, Bethard & Sumner. Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence. TACL 2014. Word alignment
SimAlign 2020 Jalili Sabet, Dufter, Yvon & Schütze. SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings. Findings of EMNLP 2020. Word alignment

License

MIT — see LICENSE.

Release files for ryokai 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ryokai 0.1.0
File Size Uploaded
ryokai-0.1.0.tar.gz 37.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for ryokai 0.1.0
File Interpreter ABI Platform
ryokai-0.1.0-py3-none-any.whl Python 3 none any Details

Total release size: 72.3 kB

Release files / ryokai-0.1.0.tar.gz

Download URL ryokai-0.1.0.tar.gz
Size 37.9 kB
Tags Source
SHA-256 checksum
How to use checksums
d2d155bc52b34bb1efbd2122d17026b75bc2e3cd7bccfa33d6bbf5faf3e4514c
BLAKE2b-256 checksum
How to use checksums
aba58bc0fee4d376c6e35e73a8553c53a51f5c9d63aefb00c31393c57a299c1a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.12

Release files / ryokai-0.1.0-py3-none-any.whl

Download URL ryokai-0.1.0-py3-none-any.whl
Size 34.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b3e722143305b3b6319b3cafd280880af8640decb26b95584ff65921e065b263
BLAKE2b-256 checksum
How to use checksums
95367f88a944e235c221203aaf0de5db020d918d7c95b400041b9e91ec31264f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.12

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page