codetransformer 0.3.0

Python code object transformers

Bytecode transformers for CPython inspired by the ast module’s NodeTransformer.

CodeTransformer API

visit_{OP}

Just like the NodeTransformer, we write visit_* methods that define how we act on an instruction.

For example (taken from my lazy library):

def visit_UNARY_NOT(self, instr):
    """
    Replace the `not` operator to act on the values that the thunks
    represent.
    This makes `not` lazy.
    """
    yield self.LOAD_CONST(_lazy_not).steal(instr)
    # TOS = _lazy_not
    # TOS1 = arg

    yield Instruction(ops.ROT_TWO)
    # TOS = arg
    # TOS1 = _lazy_not

    yield Instruction(ops.CALL_FUNCTION, 1)
    # TOS = _lazy_not(arg)

This visitor is applied to a unary not instruction (not a) and replaces it with code that is like: _lazy_not(a)

These methods will act on any opcode.

These methods are passed an Instruction object as the argument.

visit_{OTHER}

Code objects also have some data other than their bytecode. We can act on these things as well.

These methods are passed the type that occupied the given field.

1. visit_name: A transformer for the co_names field.

2. visit_varname: A transformer for the co_varnames field.

3. visit_freevar: A transformer for the co_freevars field.

4. visit_cellvar: A transformer for the co_cellvars field.

5. visit_default: A transformer for the co_defaults field.

6. visit_const: A transformer for the co_consts field.

A note about visit_const: One should be sure to call super().visit_const(const) inside of their definiton to recursivly apply your transformer to nested code objects.

const_index

One of the best uses of a bytecode transform is to make something available at runtime without putting a name in the namespace. We can do this by putting a new entry in the co_consts.

The const_index function accepts the value you want to put into the consts and returns the index as an int. This will create a new entry if needed.

The LOAD_CONST method of a CodeTransformer is a shortcut that returns a LOAD_CONST instruction object with the argument as the index of the object passed.

steal

steal is a method of the Instruction object that steals the jump target of another instruction. For example, if an instruction a is jumping to instruction b and instruction c steals b, then a will jump to b. This is useful when you are replacing an instruction with a transformer but want to preserve jumps.

Utilities

asconstants

This decorator will inline objects into a piece of code so that the names do not need to be looked up at runtime.

Example:

>>> from codetransformer import asconstants
>>> @asconstants(a=1)
>>> def f():
...     return a
...
>>> f()
1
>>> a = 5
>>> f()
1

This will work in a fresh session where a is not defined because the name a will be inlined with the constant value: 1. If a is defined, it will still be overridden with the new value.

This decorator can also take a variable amount of of builtin names:

>>> tuple = None
>>> @asconstants('tuple', 'list')
... def f(a):
...     if a:
...         return tuple
...     return list
...
>>> f(True) is tuple
False

These strings are take as the original builtin values, even if they have been overridden. These will still be faster than doing a global lookup to find the object. If no arguments are passed, it means: assume all the builtin names are constants.

with_code_transformation

This is a factory that converts CodeTransformer instances into function decorators. For example:

>>> @with_code_transformation(MyCodeTransformer())
... def f():
...    # function logic
...    ...
...

This takes binds f to a function who’s __code__ object has been transformed with an instance of MyCodeTransformer.