permped-text 0.1.0

A package for text processing.

permped-text

A Python library for preprocessing text before sending it to LLMs.

This library helps you:

• Reduce the size of your text by cleaning and minimizing it.
• Extract unique words to avoid repetition.
• Prepare text efficiently before sending it to large language models (LLMs).

Features

• get_unique_text(text) – Returns only the unique words from your text.
• minimize_text(text) – Minimizes and simplifies your text.
• clean_and_filter_text(text) – Cleans your text and removes unnecessary characters.

Installation

pip install permped-text

from permped_text import get_unique_text, minimize_text, clean_and_filter_text

text = "This is an example text. Some words repeat words."

unique_text = get_unique_text(text)
clean_text = minimize_text(unique_text)
filtered_text = clean_and_filter_text(clean_text)

print(filtered_text)