Skip to main content

Document-first text pre-processing mini-language loosely inspired by at-expressions in Racket

Project description

Paxter

Docs Documentation Status
Tests
Travis-CI Build Status Requirements Status Coverage Status Codacy Code Quality Status
Package
PyPI Package latest release PyPI Wheel Supported versions Supported implementations

Paxter is a document-first text pre-processing mini-language, loosely inspired by at-expressions in Racket.

  • The language mainly provides a way to parse an input text into a document tree (similarly to a DOM).
  • The language itself does not specify how the parsed tree will be transformed into the final rendered output text.
  • Users of this package may opt-in to use pre-defined flavors of tree transformers or choose to implement a new one by themselves.

Installation

This package can be installed from PyPI via pip command (or any other methods of your choice):

$ pip install paxter

Programmatic Usage

The package is mainly intended to be used as a library. Pre-defined tree transformers are available to be utilized right away without users having to write custom transformers.

Here is one way to use this library with Simple Snake flavor of tree transformer:

from paxter.core import Parser
from paxter.flavors import SimpleSnakeTransformer

parser = Parser()
transformer = SimpleSnakeTransformer()

env = {
    'name': "John Smith",
    'age': 47,
    'occupation': "Student",
    'strip': lambda token: token.strip(),
    'tag': lambda token, label: f"<{label}>{token}</{label}>",
}

tree = parser.parse('''\
@!##{
def add_one(num):
    return num + 1
}##\
\
@strip{   Hello, my full name is @tag[label="b"]{John Smith}   }.
I am currently @age years old, and by this time next year I will be @{age + 1} years old.
My email is @"john@example.com".
@!{age_next_year = add_one(age)}\

Do you know that 1 + 1 = @{1 + 1}?

@!##{
import itertools

counter = itertools.count(start=1)
}##\
\
Let's count: @{next(counter)}, @{next(counter)}, @{next(counter)}.
''')

updated_env, output_text = transformer.transform(env, tree)
print(f"Age next year: {updated_env['age_next_year']}")
print(output_text)

The above script will print the following text:

Age next year: 48
Hello, my full name is <b>John Smith</b>.
I am currently 47 years old, and by this time next year I will be 48 years old.
My email is john@example.com.

Do you know that 1 + 1 = 2?

Let's count: 1, 2, 3.

Library users could also write their own custom transformers by extending the paxter.core.BaseTransformer class and use it in any way they want. Stay tuned for the tutorial.

CLI Usage

While this feature is still a work in progress, users may try making a call to the following command to get started:

$ python -m paxter  # provide --help for help messages

Documentation

Future Plans

  • Experiment with different kinds of transformers and use it in real life
  • Richer experience with environments and stores (adding standard string functions, etc.)
  • Re-implementing lexers and parsers in Rust for better performance and portability to other environments (such as WASM).
  • And more!

Development

Makefile contains a lot of utility scripts.
See help command by simply running make or make help.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for paxter, version 0.3.0
Filename, size File type Python version Upload date Hashes
Filename, size paxter-0.3.0-py3-none-any.whl (29.2 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size paxter-0.3.0.tar.gz (36.7 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page