Skip to main content

morpht5

This package contains the source code for the MorphT5 models used in the Low Resource Interlinear Translation paper.

Installation

pip install morpht5

Components

Models

The package includes three model variants, incorporating morphological features through dedicated embedding layers:

  • MorphT5SumModel: Model using positional summation for combining text and morphological tag embeddings
  • MorphT5AutoModel: Base model with autoencoder-style morphological tag embeddings
  • MorphT5ConcatModel: Model using concatenation for combining text and morphological tag embeddings

Tokenizer

The package comes with a tokenizer that allows for encoding text and morphological tags into tokens:

>>> from morpht5 import MorphT5Tokenizer
>>> text = ["Λέγει", "αὐτῷ", "ὁ", "Ἰησοῦς", "Ἔγειρε", "ἆρον", "τὸν", "κράβαττόν", "σου", "καὶ", "περιπάτει"]
>>> tags = [
...     "V-PIA-3S",
...     "PPro-DM3S",
...     "Art-NMS",
...     "N-NMS",
...     "V-PMA-2S",
...     "V-AMA-2S",
...     "Art-AMS",
...     "N-AMS",
...     "PPro-G2S",
...     "Conj",
...     "V-PMA-2S",
... ]
>>> tokenizer = MorphT5Tokenizer.from_pretrained("mrapacz/interlinear-en-philta-emb-auto-diacritics-bh")
>>> inputs = tokenizer(text=text, morph_tags=tags, return_tensors="pt")
>>> inputs.keys()
dict_keys(['input_ids', 'attention_mask', 'input_morphs'])

Tagsets

The package comes with Enum classes for two morphological tagsets - one compiled from BibleHub and one compiled from Oblubienica:

from morpht5 import BibleHubTag, OblubienicaTag

Both tagsets cover comprehensive morphological features:

  • Parts of Speech (Verb, Noun, Adjective, etc.)
  • Person (1st, 2nd, 3rd)
  • Tense (Present, Imperfect, Future, Aorist, Perfect, Pluperfect)
  • Mood (Indicative, Imperative, Subjunctive, Optative, Infinitive, Participle)
  • Voice (Active, Middle, Passive)
  • Case (Nominative, Vocative, Accusative, Genitive, Dative)
  • Number (Singular, Plural)
  • Gender (Masculine, Feminine, Neuter)
  • Degree (Positive, Comparative, Superlative)

The tagsets differ in their annotation style:

  • BibleHub: Compact format (e.g., V-PIA-3S for "Verb - Present Indicative Active - 3rd Person Singular")
  • Oblubienica: Verbose format (e.g., vi Pres Act 3 Sg for the same morphological information)
>>> from morpht5 import BibleHubTag, OblubienicaTag
>>> len(BibleHubTag)
684
>>> len(OblubienicaTag)
1070

Formatting

There's also a utility function for formatting interlinear translations:

>>> from morpht5.utils.formatting import format_interlinear
>>> text = ['Λέγει', 'αὐτῷ', 'ὁ', 'Ἰησοῦς', 'Ἔγειρε', 'ἆρον', 'τὸν', 'κράβαττόν', 'σου', 'καὶ', 'περιπάτει']
>>> tags = ['V-PIA-3S', 'PPro-DM3S', 'Art-NMS', 'N-NMS', 'V-PMA-2S', 'V-AMA-2S', 'Art-AMS', 'N-AMS', 'PPro-G2S', 'Conj', 'V-PMA-2S']
>>> trans_pl = ['Mówi', 'mu', '-', 'Jezus', 'wstawaj', 'weź', '-', 'matę', 'swoją', 'i', 'chodź']
>>> trans_en = ['says', 'to him', '-', 'jesus', 'arise', 'take up', '-', 'mat', 'of you', 'and', 'walk']
>>> print(format_interlinear(text, tags, trans_en, trans_pl))
 Λέγει   |    αὐτῷ   |    ὁ    | Ἰησοῦς |  Ἔγειρε  |   ἆρον   |   τὸν   | κράβαττόν |   σου    | καὶ  | περιπάτει
V-PIA-3S | PPro-DM3S | Art-NMS | N-NMS  | V-PMA-2S | V-AMA-2S | Art-AMS |   N-AMS   | PPro-G2S | Conj |  V-PMA-2S
  says   |   to him  |    -    | jesus  |  arise   | take up  |   the   |    mat    |  of you  | and  |    walk
  Mówi   |     mu    |    -    | Jezus  | wstawaj  |   weź    |    -    |    matę   |  swoją   |  i   |   chodź

License

This project is licensed under the MIT License. See the LICENSE file for details.

Metadata

Release files for morpht5 0.2.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for morpht5 0.2.2
File Size Uploaded Provenance Actions
morpht5-0.2.2.tar.gz 70.3 kB Without provenance Details for morpht5-0.2.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for morpht5 0.2.2
File Interpreter ABI Platform Actions
morpht5-0.2.2-py3-none-any.whl Python 3 none any Details for morpht5-0.2.2-py3-none-any.whl

Total release size: 142.5 kB

Release files / morpht5-0.2.2.tar.gz

Download URL morpht5-0.2.2.tar.gz
Size 70.3 kB
Tags Source
SHA-256 checksum
How to use checksums
c40202f08927cae7137a6ede7ded369c0a29c69ad40fa7b268fca0484d68e1dc
BLAKE2b-256 checksum
How to use checksums
8c26bb8ae2e12658363431eeb5105502c25031e5da4a8d259ddbeb47e039770f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.4 CPython/3.12.7 Darwin/23.2.0

Release files / morpht5-0.2.2-py3-none-any.whl

Download URL morpht5-0.2.2-py3-none-any.whl
Size 72.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c1f882826ef94896011e311805941c526a0d34af7a0de864b989f4777af89168
BLAKE2b-256 checksum
How to use checksums
d48cc08a95554fb0a7abd8a17522d22dc10bb4e2622163c0b9624ad113defc9c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.4 CPython/3.12.7 Darwin/23.2.0

Security for morpht5 0.2.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page