paxter 0.3.0

Document-first text pre-processing mini-language loosely inspired by at-expressions in Racket

Paxter

Docs
Tests
Package

Paxter is a document-first text pre-processing mini-language, loosely inspired by at-expressions in Racket.

• The language mainly provides a way to parse an input text into a document tree (similarly to a DOM).
• The language itself does not specify how the parsed tree will be transformed into the final rendered output text.
• Users of this package may opt-in to use pre-defined flavors of tree transformers or choose to implement a new one by themselves.

Installation

This package can be installed from PyPI via pip command (or any other methods of your choice):

$ pip install paxter

Programmatic Usage

The package is mainly intended to be used as a library. Pre-defined tree transformers are available to be utilized right away without users having to write custom transformers.

Here is one way to use this library with Simple Snake flavor of tree transformer:

from paxter.core import Parser
from paxter.flavors import SimpleSnakeTransformer

parser = Parser()
transformer = SimpleSnakeTransformer()

env = {
    'name': "John Smith",
    'age': 47,
    'occupation': "Student",
    'strip': lambda token: token.strip(),
    'tag': lambda token, label: f"<{label}>{token}</{label}>",
}

tree = parser.parse('''\
@!##{
def add_one(num):
    return num + 1
}##\
\
@strip{   Hello, my full name is @tag[label="b"]{John Smith}   }.
I am currently @age years old, and by this time next year I will be @{age + 1} years old.
My email is @"[e-mail]".
@!{age_next_year = add_one(age)}\

Do you know that 1 + 1 = @{1 + 1}?

@!##{
import itertools

counter = itertools.count(start=1)
}##\
\
Let's count: @{next(counter)}, @{next(counter)}, @{next(counter)}.
''')

updated_env, output_text = transformer.transform(env, tree)
print(f"Age next year: {updated_env['age_next_year']}")
print(output_text)

The above script will print the following text:

Age next year: 48
Hello, my full name is <b>John Smith</b>.
I am currently 47 years old, and by this time next year I will be 48 years old.
My email is [e-mail].

Do you know that 1 + 1 = 2?

Let's count: 1, 2, 3.

Library users could also write their own custom transformers by extending the paxter.core.BaseTransformer class and use it in any way they want. Stay tuned for the tutorial.

CLI Usage

While this feature is still a work in progress, users may try making a call to the following command to get started:

$ python -m paxter  # provide --help for help messages

Documentation

• Rough description of Paxter grammar
• ReadTheDocs documentation (under construction)
• Paxter documentation can be generated with pdoc3 which can be installed with pip install pdoc3. Then preview this package with pdoc --html : paxter or compiled with pdoc --html paxter.

Future Plans

• Experiment with different kinds of transformers and use it in real life
• Richer experience with environments and stores (adding standard string functions, etc.)
• Re-implementing lexers and parsers in Rust for better performance and portability to other environments (such as WASM).
• And more!

Development

Makefile contains a lot of utility scripts.
See help command by simply running make or make help.