gekkota 0.6.0

Python code-generation for Python

This is a Python code-generation module.

• Generates any Python statement/expression
• Places parens to ensure expression priorities are unchanged
• Places extra newlines before/after class/function definitions to conform with PEP 8
• 100% coverage of type hints, passing strict Pyright checks on every PR
• Meaningful type hierarchy inspired by Python grammar to guarantee syntax validity almost in every case.
• Covered with diamonds tests completely
• Compatible with wordstreamer renderables via a wrapper

Installation

Just install gekkota package, e.g. with python -m pip install gekkota (or any other package manager of your choice)

Rendering and configuration

To render any Renderable into a string, you could use a few approaches:

• str(renderable): renders a Renderable with default configuration (check below)
• renderable.render_str(): also default configuration
• renderable.render_str(config): overrides default config options with provided in config mapping. Unspecified keys remain at default values

Here is current default config:

default_config: Config = {
    "tab_size": 4,  # how much chars to use in indentation
    "compact": False,  # if True, renders without redundant whitespace (e.g "for [i, e] in enumerate(a)" renders as "for[i,e]in enumerate(a)")
    "tab_char": " ",  # character used for indentation
    # "place_semicolons" and "inline_small_stmts" options have some performance impact, since those require checking for newlines in token stream before re-streaming tokens.
    # this impact is probably negligible, but be aware of it
    "place_semicolons": False,  # if True, semicolons are placed after one-line statements
    "inline_small_stmts": False,  # if True, one-line statements are inlined. Overrides "place_semicolons" if True.
}

Expressions

Basic expressions

Your starting points would be to_expression and Name:

from gekkota import Name, to_expression

# Name(name: str, annotation: Optional[Expression] = None)
# to_expression(value: int | float | complex | str | bytes | bool | None)

a = Name("a")
b = Name("b")
six = to_expression(6)

# prints 'a + b * 6'
print(
    (a + b * six)
)

Name, as many other classes in the module, is an Expression instance

Expressions support most operations to combine with other expressions.
Exceptions are:

• Attribute reference: for that you should use Expression.getattr(other: str)
• Indexing: Expression.index(index: Expression)
• Slicing: Expression.index(index: Union[SliceExpr, Sequence[SliceExpr]])
• Equality / Inequality: Expression.eq(right_hand_side) and Expression.neq(right_hand_side) respectively
• is: Expression.is_(right_hand_side),
• is not: Expression.is_not(right_hand_side)
• in: Expression.in_(right_hand_side)
• not in: Expression.not_in(right_hand_side)
• and: Expression.and_(right_hand_side)
• or: Expression.or_(right_hand_side)
• await: Expression.await_()
• := assignment: Expression.assign(value)
• Ternary operator: Expression.if_(condition, alternative)

For example:

from gekkota import Name

a = Name("a")
b = Name("b")

expression = a.await_().in_(b)

print(expression) # await a in b

For any other operation on expressions you can just use familiar Python syntax:

from gekkota import Name

a = Name("a")
b = Name("b")
c = Name("c")

print(
    (a + b * c / a(b, c)) # 'a + b * c / a(b, c)'
)

Sequences

Most convenient way to create sequence literals is, again, to_expression:

from gekkota import to_expression, Name

a = Name("a")
b = Name("b")

print(
    to_expression( (a, b, 6) ), # '(a, b, 6)' (notice that to_expression is recursive)
    to_expression( (a, ) ),     # '(a, )'
    to_expression([a, b]),      # '[a, b]'
    to_expression([]),          # '[]'
    to_expression({a: b}),      # '{a: b}'
    to_expression(set()),       # 'set()'
    to_expression([a, [a, b]]), # '[a, [a, b]]'
)

If you want to have more precise control, you can use TupleExpr, ListExpr, SetExpr and DictExpr for this. All have same constructor signature: (values: Sequence[Expression]) (except DictExpr, which has KeyValue values)

To create comprehensions:

from gekkota import Name, GeneratorFor, GeneratorIf
from gekkota import (
    ListComprehension,
    DictComprehension,
    KeyValue,
    SetComprehension, # same usage as ListComprehension
)

a, b, c, d = map(Name, "abcd")

# ListComprehension(generator_or_expr: GeneratorBase | Expression, parts: Sequence[GeneratorPart] = ())
print(
    ListComprehension(
        a,
        [
            # GeneratorFor(target: AssignmentTarget, iterator: Expression, *, is_async: bool = False)
            GeneratorFor(b, c),

            # GeneratorIf(condition: Expression)
            GeneratorIf(b.eq(d))
        ]
    )
) # [a for b in c if b == d]

# DictComprehension(generator_or_expr: GeneratorBase | KeyValue, parts: Sequence[GeneratorPart] = ())
# GeneratorPart == GeneratorFor | GeneratorIf
print(
    DictComprehension(KeyValue(a, b), [GeneratorFor(c, d), GeneratorIf(b.eq(d))])
) # {a: b for c in d if b == d}

Keyword call args

Use CallArg to provide keyword call args:

from gekkota import Name, to_expression

print_ = Name("print")

# CallArg(name: str, value: Optional[Expression] = None)
print(
    print_(
        Name("a"),
        CallArg("b"),
        CallArg("sep", to_expression(", "))
    )
) # print(a, b, sep=', ')

Type hints

To annotate a name, just pass an additional parameter to Name:

from gekkota import Name

a = Name("a", Name("int"))

print(a) # a: int

New in version 0.6: Name is now generic, depending on the presence of the annotation.
Some constructors now accept only annotated Name, while some others accept only unannotated (depends on syntax features allowed).
This allows to ensure type soundness without breaking changes in API.

Statements

To render program code (with multiple statements), use Code:

from gekkota import Code, Assignment, Name

a = Name("a")

six = Literal(6)

create_variable = Assignment(
    [Name("a")],
    six + six
)

print_variable = Name("print")(a)

print(
    Code([
        create_variable,
        print_variable,
    ])
)
# prints:
# a = 6 + 6
# print(a)

To render a block of code, use Block:

from gekkota import Block, IfStmt, Assignment, Name

a = Name("a")
b = Name("b")

six = Literal(6)

create_variable = Assignment(
    [Name("a")],
    six + six
)

print_variable = Name("print")(a)

print(
    IfStmt(
        b,
        Block([
            create_variable,
            print_variable,
        ])
    )
)
# prints:
# if b:
#     a = 6 + 6
#     print(a)

If the difference between two is not obvious: Code just renders statements on separate lines, while block also adds a newline before the first statement and indentation to every line. Moreover, Code([]) renders into "", while Block([]) — into "\n pass"

Small statements

Here is an example of a few small statements:

from gekkota import Name, SequenceExpr
from gekkota import (
    ReturnStmt,
    DelStmt,
    AssertStmt,
    BreakStmt,
    ContinueStmt,
    YieldStmt,
    YieldFromStmt,
    NonLocalStmt,
    GlobalStmt,
    PassStmt,
    RaiseStmt,
    AsyncStmt
)

a, b, c = map(Name, "abc")

print(ReturnStmt(a)) # 'return a'

print(YieldStmt(a)) # 'yield a'
print(YieldFromStmt(b)) # 'yield from b'

print(DelStmt(a, b)) # 'del a, b'

print(AssertStmt(a)) # 'assert a'

print(BreakStmt()) # 'break'

print(ContinueStmt()) # 'continue'

print(GlobalStmt(a, b)) # 'global a, b'
print(NonLocalStmt(a, b)) # 'nonlocal a, b'

print(PassStmt()) # 'pass'

print(RaiseStmt()) # 'raise'
print(RaiseStmt(a)) # 'raise a'
print(RaiseStmt(a, b)) # 'raise a from b'

Type statement

Gekkota 0.6 supports PEP 695 which adds new syntax, including type statement:

type Point = tuple[float, float]

To generate such statement in gekkota, use gekkota.annotations.TypeStmt:

from gekkota import TypeStmt

# type Point = tuple[float, float]
print(
    TypeStmt(
        "Point",
        [],
        TupleExpr([Name("float"), Name("float")])
    )
)

The second argument is a sequence of type parameters (for genric types): Sequence[TypeParam]

You can use several types as TypeParam:

from gekkota import TypeParam, TypeStmt, Name, TypeVarParam, TypeVarTupleParam, ParamSpecParam

def make_type(type_params: Sequence[TypeParam]):
    return TypeStmt(
        "CoolAlias",
        type_params,
        Name("int")
    )


print(make_type([])) # type CoolAlias = int

# (both) type CoolAlias[T] = int
print(make_type([Name("T")]))
print(
    make_type([TypeVarParam(Name("T"))])
)

# (both) type CoolAlias[T: float] = int
print(make_type([Name("T", Name("float"))]))
print(
    make_type(
        [
            TypeVarParam(
                name=Name("T"),
                value=Name("float")
            )
        ]
    )
)

# type CoolAlias[*T] = int
print(
    make_type(
        [
            TypeVarTupleParam(
                Name("T"),
            )
        ]
    )
)

# type CoolAlias[**T] = int
print(
    make_type(
        [
            ParamSpecParam(
                Name("T"),
            )
        ]
    )
)

Generic Functions and Classes

FuncDef and ClassDef now accept an optional keyword argument type_params accepting Sequence[TypeParam]:

from gekkota import FuncDef, Name, PassStmt

typeparam = Name("T")

newfunc = FuncDef(
    name="newfunc",
    args=[Name("x", typeparam)],
    body=PassStmt(),
    rtype=typeparam,
    type_params=[typeparam],
)

print(newfunc)
"""
def newfunc[T](x: T) -> T: pass
"""

Assignment

For common assigment use Assignment:

from gekkota import Assignment, Name

a, b, c = map(Name, "abc")

# Assignment(targets: Sequence[AssignmentTarget] | AnnotatedTarget, value: Expression)

print(
    Assignment([a], b), # a = b
    Assignment([a.index(b)], c) # a[b] = c
    Assignment([a, b], c), # a = b = c
)

To annotate assignment (or just annotate a variable), use AnnotatedTarget:

from gekkota import Assignment, AnnotatedTarget, Name

a, b, c = map(Name, "abc")
D = Name("D")

# AnnotatedTarget(target: AssignmentTarget, annotation: Expression)
print(
    Assignment(AnnotatedTarget(a, D), b), # a: D = b
    Assignment(AnnotatedTarget(a.index(b), D), c) # a[b]: D = c
    Assignment([a, b], c), # a = b = c
)

For augmented assignment (e.g. +=) use AugmentedAssignment:

from gekkota import Assignment, Name

a, b, c = map(Name, "abc")

# AugmentedAssignment(target: AugAssignmentTarget, op: str, expression: Expression)

print(
    AugmentedAssignment(a, "+=", b), # a += b
    AugmentedAssignment(a.index(b), "*=", c) # a *= c
)

Control flow

For control flow you can use IfStmt, ElifStmt and ElseStmt:

from gekkota import Name, IfStmt, ElifStmt, ElseStmt, Code

a, b, c = map(Name, "abc")

# IfStmt(condition: Expression, body: Statement)
# ElifStmt(condition: Expression, body: Statement)
# ElseStmt(body: Statement)
code = Code([
    IfStmt(a, b),
    ElifStmt(b, a),
    ElseStmt(c)
])

print(code)
"""
if a: b
elif b: a
else: c
"""

Loops

Use ForStmt and WhileStmt for loops:

from gekkota import ForStmt, WhileStmt, Name

a, b, c = map(Name, "abc")

# ForStmt(target: Expression, iterator: Expression, body: Statement, *, is_async: bool = False)
print(
    ForStmt(a, b, c)
) # for a in b: c

# WhileStmt(condition: Expression, body: Statement)
print(
    WhileStmt(a, b)
) # while a: b

Functions

To render a function definition, you will need a FuncDef:

from gekkota import Name, FuncDef

a, b, c = map(Name, "abc")

# FuncDef(
#    name: str,
#    args: Sequence[FuncArg],
#    body: Statement,
#    *,
#    rtype: Optional[Expression] = None,
#    is_async: bool = False,
#    type_params: Sequence[TypeParam] = (),
# )

print(
    FuncDef(
        "cool_func",
        [a],
        b,
        rtype=c,
    )
) # def cool_func(a) -> c: b

New in 0.6: Now FuncDef accepts a type_params argument (check Generic Functions and Classes)

To provide a default value and/or annotations to arguments, use FuncArg:

from gekkota import Name, FuncDef, FuncArg, to_expression

a, b, c = map(Name, "abc")

# FuncDef(name: str, args: Sequence[FuncArg], body: Statement, *, rtype: Optional[Expression] = None, is_async: bool = False)
# FuncArg(
#     name: str,
#     annotation: Optional[Expression] = None,
#     default_value: Optional[Expression] = None,
#     *,
#     late_bound_default: bool = False,
# )
print(
    FuncDef(
        "cool_func",
        [
            FuncArg(
                "a",
                Name("int"),
                to_expression(0)
            )
        ],
        b,
        rtype=c,
    )
) # def cool_func(a: int = 0) -> c: b

New in version 0.6: support for PEP 671 (Draft) – Syntax for late-bound function argument defaults

Other argument types are:

• StarArg(value: T = None): generates *value, * by default
• DoubleStarArg(value): same as StarArg, but with **
• Slash() is / (a mark of positional-only arguments in Python 3.8+)

Lambda functions are generated using LambDef:

from gekkota import Name, LambDef

a, b, c = map(Name, "abc")

# LambDef(args: Sequence[FuncArg], body: Expression)
print(
    LambDef(
        [a],
        b,
    )
) # lambda a: b

To decorate a function/class, use Decorated:

from gekkota import Name, FuncDef, Decorated

decorator = Name("decorator")
a, b, c = map(Name, "abc")

# Decorated(decorator: Expression, statement: ClassDef | FuncDef)
# FuncDef(name: str, args: Sequence[FuncArg], body: Statement, *, rtype: Optional[Expression] = None, is_async: bool = False)
print(
    Decorated(
        decorator,
        FuncDef(
            "cool_func",
            [a],
            b,
            rtype=c,
        )
    )
)
# @decorator
# def cool_func(a) -> c: b

Classes

To define a class, use ClassDef:

from gekkota import Name, ClassDef

a, b, c = map(Name, "abc")

# ClassDef(
#     name: str,
#     args: Sequence[CallArg | Expression],
#     body: Statement,
#     *,
#     type_params: Sequence[TypeParam] = (),
# )
print(
    ClassDef("MyClass1", [], a)
) # class MyClass1: a

print(
    ClassDef("MyClass2", [b], c)
) # class MyClass2(b): c

New in 0.6: Now ClassDef accepts a type_params argument (check Generic Functions and Classes)

Imports

To render imports, use ImportStmt and FromImportStmt:

from gekkota import Name, StarArg, ImportDots, ImportSource, ImportStmt, FromImportStmt, ImportAlias


# ImportStmt(names: Sequence[ImportAlias | Name | StarArg[None]])
print(
    ImportStmt([Name("a")])
) # import a

print(
    ImportStmt([Name("a"), Name("b")])
) # import a, b

# FromImportStmt(source: ImportSource | Name, names: Sequence[ImportAlias | Name | StarArg[None]])
# ImportAlias(name: Name, alias: Name | None = None)
print(
    FromImportStmt(
        Name("math"),
        [
            Name("cos"),
            ImportAlias(Name("sin"), Name("tan")) # we do a little trolling
        ]
    )
) # from math import cos, sin as tan

print(
    FromImportStmt(
        Name("gekkota"),
        [StarArg()]
    )
) # from gekkota import *

# ImportDots(length: int = 1)
print(
    FromImportStmt(
        ImportDots(),
        [StarArg()]
    )
) # from . import *

# ImportSource(parts: Sequence[str])
print(
    FromImportStmt(
        ImportSource(["", "values"]),
        [Name("Name")]
    )
) # from .values import Name

Exceptions

from gekkota import Name, TryStmt, ExceptStmt, FinallyStmt, Block

a, b, e = map(Name, "abe")

# TryStmt(body: Statement)
print(
    TryStmt(a)
) # try: a

# ExceptStmt(exceptions: Sequence[Expression] | None, alias: Name | None, body: Statement)
print(
    ExceptStmt(None, None, a)
) # except: a

print(
    ExceptStmt(None, None, Block([]))
)
# except:
#     pass

print(
    ExceptStmt([a], None, b)
) # except a: b

print(
    ExceptStmt([a], e, b)
) # except a as e: b

# FinallyStmt(body: Statement)
print(
    FinallyStmt(a)
) # finally: a

Context Managers

from gekkota import Name, WithStmt, WithTarget

a, b, e = map(Name, "abe")

# WithStmt(targets: Sequence[WithTarget | Expression], body: Statement, *, is_async: bool = False,)
print(
    WithStmt([a], b)
) # with a: b

# WithTarget(expression: Expression, alias: str | None = None)
print(
    WithStmt([WithTarget(a, "aaaa")], b)
) # with a as aaaa: b

print(
    WithStmt([a], b, is_async=True)
) # async with a: b

Pattern matching

This section is currently unfinished, check pattern_matching.py

Custom rendering

If your custom element can be meaningfully represented as a combination of existing elements, you can use a function instead of a class:

from gekkota import Expression

def Square(e: Expression) -> Expression:
    return e * e

This is a pretty obvious approach, but often it works best.

---

While being aimed at Python code generation, gekkota is pretty extensible, and can be used to render different things.
You can build custom renderables, statements, expressions, and so on.

The simplest example of a custom renderable would be:

from gekkota import Renderable, StrGen, Config


class RenderString(Renderable):
    """It renders whatever is passed to it"""

    def __init__(self, value: str):
        self.value = value

    def render(self, config: Config) -> StrGen:
        yield self.value

Let's suppose you want to render a custom expression: a custom sequence literal (obviously isn't valid in Python, but you need it for some reason).
Suppose your custom literal would be in form of <|value1, value2, ...|>.

You can extend SequenceExpr for that:

from gekkota import SequenceExpr, Name

class MyCoolSequence(SequenceExpr):
    parens = "<|", "|>"


seq = MyCoolSequence([Name("a"), Name("b")])

print(seq) # <|a,b|>

That's it, you're ready to render this literal (which, again, isn't valid in Python but anyway).

Or you could go further and write rendering by yourself (it's easier than it sounds):

from gekkota import Expression, Config


class MyCoolSequence(Expression):
    def __init__(self, values: Sequence[Expression]):
        self.values = values

    # could be rewritten to be simpler, check `Useful utils` section below
    def render(self, config: Config) -> StrGen:
        yield "<|"

        for i, item in enumerate(self.values):
            yield from item.render(config)

            if i + 1 < len(self.values): # no comma after last element
                yield ","
                yield " "

        yield "|>"

It's fairly easy, just render every part in the right order:

• To render a string, use yield string
• To render a Renderable, use yield from renderable.render(config)

Choosing a right base class

To choose a right base class, think in what context you want to use your renderable.
If there is a similar context in Python (e.g. your renderable is a block statement, like for or if), extend that class.

After choosing a right base class, check if it has a predefined render, maybe you won't need to write everything by yourself.
For example, with BlockStmt you need to provide render_head instead:

# that's the actual source from module, not an example

class BlockStmt(Statement):
    body: Statement

    def render_head(self, config: Config) -> StrGen:
        return NotImplemented

    def render(self, config: Config) -> StrGen:
        yield from self.render_head(config)
        yield ":"
        yield " "
        yield from self.body.render(config)

[...]

class ElseStmt(BlockStmt):
    def __init__(self, body: Statement):
        self.body = body

    def render_head(self, config: config) -> StrGen:
        yield "else"

Useful utils

gekkota.utils provides Utils class which is useful for custom renderables. For example, custom MyCoolSequence could be implemented as:

from gekkota import Expression, Utils


class MyCoolSequence(Expression):
    def __init__(self, values: Sequence[Expression]):
        self.values = values

    def render(self, config: config) -> StrGen:
        yield from Utils.wrap(
            ["<|", "|>"],
            Utils.comma_separated(self.values, config)
        )

Methods provided in Utils:

• add_tab(generator: StrGen, config: Config) -> StrGen
  Adds indentation to a stream of tokens, using provided config
  For example, Utils.add_tab(Name("a").render(config), config) -> Iterable of [' ', 'a']

• separated(separator: Sequence[str], renderables: Sequence[Renderable], config: Config) -> StrGen
  Inserts separator between renderables (and renders them in stream)
  For example: Utils.separated([",", " "], self.values, config) - inserts ", " between elements of self.values

• separated_str(separator: Sequence[str], strings: Sequence[str], config: Config)
  Same as previous, but for str sequences

• comma_separated(renderables: Sequence[Renderable], config: Config) -> StrGen
  Alias for Utils.separated([",", " "], renderables, config)

• make_compact(generator: StrGen, config: Config) -> StrGen
  Filters all unneccessary whitespace from stream (doesn't respect config["compact"]). Config is unused at the moment, but provided for compatibility with future updates

• wrap(parens: Sequence[str], generator: StrGen) -> StrGen
  Wraps a token stream with strings from parens array (should have 2 elements).
  In other words, inserts parens[0] at the start of the stream, and parens[1] at the end

wordstreamer compatibility

gekkota contains experimental wordstreamer compatibility layer to use gekkota objects in wordstreamer and vice versa.

Use gekkota.Renderable where wordstreamer.Renderable can be used:

from gekkota import WSRenderable, Literal

ws_compatible = WSRenderable(Literal(6)) # this is a wordstreamer.Renderable now

Use wordstreamer.Renderable where gekkota.Renderable can be used:

from gekkota import WStoG, Literal

from wordstreamer.startkit import Stringify

# create a class to define the type of renderable
class WSLiteral(WStoG, Literal):
    pass

some_number = Stringify(6)

print(WSLiteral(some_number) + Literal(8)) # "6 + 8"

Insert a wordstreamer.Renderable in a string (gekkota.Literal)

from gekkota import WSString, Literal

from wordstreamer.startkit import Stringify

some_number = Stringify(6)

print(WSString(some_number)) # '"""6"""'