Toolbox for working with the Python AST
pip install ast_tools
Useful References
Passes
ast_tools provides a number of passes for rewriting function and classes (could
also work at the module level however no such pass exists). Passes are
applied with the apply_passes decorator:
@apply_passes([pass1(), pass2()])
def foo(...): ...
Each pass takes as arguments an AST, an environment, and metadata and
returns (possibly) modified versions of each.
apply_passes begins a chain of rewrites by first looking
up the ast of the decorated object and gather attempts to gather locals
and globals from the call site to build the environment.
After all rewrites have run apply_passes serializes and
execute the rewritten ast.
Know Issues
Collecting the AST
apply_passes relies on inspect.getsource to get the
source of the decorated definition (which is then parsed to get the initial ast).
However, inspect.getsource has many limitations.
Collecting the Environment
apply_passes does its best to infer the environment
however there is no way to do this in a fully correct way. Users are
encouraged to pass environment explicitly:
@apply_passes(..., env=SymbolTable(locals(), globals()))
def foo(...): ...
Wrapping the apply_passes decorator
The apply_passes decorator must not be wrapped.
As decorators are a part of the AST of the object they are applied to they must be removed from the rewritten AST before it is executed. If they are not removed rewrites will recurse infinitely as
@apply_passes([...])
def foo(...): ...
would become
exec('''\
@apply_passes([...])
def rewritten_foo(...): ...
''')
Note: this would invoke apply_passes([...]) on rewritten_foo
To avoid this the apply_passes decorator filters itself from the decorator list. If, however,
the decorator is wrapped inside another decorator, this will fail.
Inner decorators are called multiple times
Decorators that are applied before a rewrite group will be called multiple times. See https://github.com/leonardt/ast_tools/issues/46 for detailed explanation. To avoid this users are encouraged to make rewrites the inner most decorators when possible.
Macros
Loop Unrolling
Unroll loops using the pattern
for <var> in ast_tools.macros.unroll(<iter>):
...
<iter> should be an iterable object that produces integers (e.g. range(8))
that can be evaluated at definition time (can refer to variables in the scope
of the function definition)
For example,
from ast_tools.passes import apply_passes, loop_unroll
@apply_passes([loop_unroll()])
def foo():
for i in ast_tools.macros.unroll(range(8)):
print(i)
is rewritten into
def foo():
print(0)
print(1)
print(2)
print(3)
print(4)
print(5)
print(6)
print(7)
You can also use a list of ints, here's an example that also uses a reference
to a variable defined in the outer scope:
from ast_tools.passes import apply_passes, loop_unroll
j = [1, 2, 3]
@apply_passes([loop_unroll()])
def foo():
for i in ast_tools.macros.unroll(j):
print(i)
becomes
def foo():
print(1)
print(2)
print(3)
Inlining If Statements
This macro allows you to evaluate if statements at function definition time,
so the resulting rewritten function will have the if statements marked
"inlined" removed from the final code and replaced with the chosen branch based
on evaluating the condition in the definition's enclosing scope. if
statements are marked by using the form if inline(...): where inline is
imported from the ast_tools.macros package. if statements not matching
this pattern will be ignored by the rewrite logic.
Here's an example
from ast_tools.macros import inline
from ast_tools.passes import apply_passes, if_inline
y = True
@apply_passes([if_inline()])
def foo(x):
if inline(y):
return x + 1
else:
return x - 1
import inspect
assert inspect.getsource(foo) == f"""\
def foo(x):
return x + 1
"""
1 maintainer
Metadata
1 maintainer
Release files for ast-tools 0.1.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| ast_tools-0.1.8-py38-none-any.whl | Python 3.8 | none | any | Details |
| ast_tools-0.1.8-py37-none-any.whl | Python 3.7 | none | any | Details |
| ast_tools-0.1.8-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 130.6 kB
Release files / ast_tools-0.1.8-py38-none-any.whl
| Download URL | ast_tools-0.1.8-py38-none-any.whl |
|---|---|
| Size | 43.6 kB |
| Tags | Python 3.8 |
|
SHA-256 checksum How to use checksums |
4553b3e323cc96e9677d519d3dd9566e9decb9be91902a1f7ba51ad96f8100a9
|
|
BLAKE2b-256 checksum How to use checksums |
c2de4f1a758137bdc553ac16e119c9163fc02e21aeb2883b9e8dfefed1f4b7fe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.7
|
Release files / ast_tools-0.1.8-py37-none-any.whl
| Download URL | ast_tools-0.1.8-py37-none-any.whl |
|---|---|
| Size | 43.4 kB |
| Tags | Python 3.7 |
|
SHA-256 checksum How to use checksums |
eaedb05302793a0c69484ad184f0ab577443967bc3ca51cdb6b1be3b82780f91
|
|
BLAKE2b-256 checksum How to use checksums |
483751c83e4b1b894d7d95037e1c5d5ee0537d712704f8952b69c8e2fd71b386
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.7.1
|
Release files / ast_tools-0.1.8-py3-none-any.whl
| Download URL | ast_tools-0.1.8-py3-none-any.whl |
|---|---|
| Size | 43.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bf4985821e76c236c6201072f629c8ba9a67b0aeaa094d6b260a4db2d738f424
|
|
BLAKE2b-256 checksum How to use checksums |
ca8f9b7cfe12920638baf26325f1285915db058ed9e85b9f9308af96523322af
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.1
|
