fragmentshot 0.1.4

Prompt generation for Fragment-Shot and Pivoted Fragment-Shot translation using LLMs

FragmentShot Retriever

A Python package for retrieving exemplary translations for text based on fragments from parallel corpora.

Features

• Fragment extraction from source and target texts
• Configurable maximum fragment size
• Option to enable or disable fragment overlaps
• Easy integration for retrieval workflows

Installation

You can install this package locally:

pip install fragmentshot

Or clone the repo and install in editable mode:

git clone https://github.com/schtailmuel/fragmentshot.git
cd fragmentshot
pip install -e .

Usage

from fragmentshot.retriever import FragmentShotsRetriever

src_texts = [
    "this is a sample source sentence.",
    "another example source sentence."
]

tgt_texts = [
    "dies ist ein Beispiel im Zieltext.",
    "noch ein Beispiel."
]

retriever = FragmentShotsRetriever(src_texts, tgt_texts, max_fragment_size=5, overlaps=False)

result = retriever.get_fragment_shots("The source of this is unknown.")
print(result)

Result

{
  "shots": [
    {
      "index": 1,
      "fragment": "source",
      "examples": [
        {
          "src_text": "this is a sample source sentence.",
          "tgt_text": "dies ist ein Beispiel im Zieltext."
        },
        {
          "src_text": "another example source sentence.",
          "tgt_text": "noch ein Beispiel."
        }
      ]
    },
    {
      "index": 3,
      "fragment": "this is",
      "examples": [
        {
          "src_text": "this is a sample source sentence.",
          "tgt_text": "dies ist ein Beispiel im Zieltext."
        }
      ]
    }
  ],
  "num_words": 6,
  "unknown": [
    "The",
    "of",
    "unknown"
  ]
}

Testing

Run unit tests with:

python -m unittest discover tests

License

This project is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.

Created by Samuel Frontull