reannotate 0.1.3

An extension to annotationlib to assist in creating new annotate functions

Reannotate

This library acts as an extension to the new deferred annotations that arrived as part of PEP-649/749 in Python 3.14.

Its main purpose is to make it possible to manipulate PEP-649/749 annotations in order to recreate __annotate__ functions that support all of the new annotations formats. It should be as simple to manipulate and change annotations to create new __annotate__ functions with reannotate as it was to manipulate and create new __annotations__ under older versions of Python.

It also makes it easy to retrieve annotations and evaluate them individually.

Unlike Format.FORWARDREF, get_deferred_annotations will always return DeferredAnnotation objects as the values of the annotations dictionary.

Usage

Retrieving deferred annotations

get_deferred_annotations is provided to retrieve deferred annotations from an annotated object:

from pprint import pp
from reannotate import get_deferred_annotations

class Example:
    a: int
    b: list[unknown]
    c: str | undefined

annos = get_deferred_annotations(Example)

pp(annos)

{'a': DeferredAnnotation('int'),
 'b': DeferredAnnotation('list[unknown]'),
 'c': DeferredAnnotation('str | undefined')}

To use the DeferredAnnotation objects, they have an .evaluate() method that supports the standard annotationlib formats:

from annotationlib import Format

print(annos['a'].evaluate(format=Format.VALUE))
print(annos['b'].evaluate(format=Format.FORWARDREF))
print(annos['c'].evaluate(format=Format.STRING))

<class 'int'>
list[ForwardRef('unknown', is_class=True, owner=<class '__main__.Example'>)]
str | undefined

If a value is defined at a later point, the annotation can then be evaluated fully.

unknown = float

print(annos['b'].evaluate())
print(annos['b'].is_resolved)  # True if a DeferredAnnotation has been fully evaluated

list[float]
True

Creating a new __annotate__ callable

Instances of the ReAnnotate class are intended to act as __annotate__ callables.

from annotationlib import call_annotate_function, Format
from reannotate import get_deferred_annotations, ReAnnotate

class Example:
    a: int
    b: list[undefined]

annos = get_deferred_annotations(Example)

new_annos = ReAnnotate(annos)

print(call_annotate_function(new_annos, format=Format.FORWARDREF))

{'a': <class 'int'>, 'b': list[ForwardRef('undefined', is_class=True, owner=<class '__main__.Example'>)]}

Handling Unions and Generics with forward references

reannotate provides get_origin and get_args functions, analogous to those provided by typing that can get the origin and arguments of genericised annotations even if there are forward references.

Unlike typing the objects will be returned in DeferredAnnotation format. This allows for some of them to be forward references.

Note: This relies on the assumption that the objects in question are types, and as such | indicates a union

from reannotate import get_deferred_annotations, get_origin, get_args

class Example:
    a: undefined | bytes | str
    b: unknown[str]

annos = get_deferred_annotations(Example)
a_anno = annos['a']
b_anno = annos['b']

print(get_origin(a_anno))
print(get_args(a_anno))
print()

print(get_origin(b_anno))
print(get_args(b_anno))

DeferredAnnotation('typing.Union')
(DeferredAnnotation('undefined'), DeferredAnnotation('bytes'), DeferredAnnotation('str'))

DeferredAnnotation('unknown')
(DeferredAnnotation('str'),)

The primary purpose of these functions is to allow for extracting arguments from generics to create new annotations. For example, using the argument to InitVar as the annotation for __init__ in something like dataclasses.

How does this differ from Format.FORWARDREF

Resolution

The FORWARDREF format always attempts to resolve annotations at runtime as far as possible, this means that the ForwardRef objects can be contained inside other objects and made more difficult to resolve. This resolution makes them unsuitable to use to generate new __annotate__ callables.

from annotationlib import get_annotations, Format
from reannotate import get_deferred_annotations

class Example:
    a: list[ref]

print(get_annotations(Example, format=Format.FORWARDREF)['a'])
print(get_deferred_annotations(Example)['a'])

list[ForwardRef('ref', is_class=True, owner=<class '__main__.Example'>)]
DeferredAnnotation('list[ref]')

In this case if ref is defined later, the DeferredAnnotation can be resolved using .evaluate(), but resolving the annotation from the ForwardRef format requires evaluating the reference inside the GenericAlias for list.

DeferredAnnotation also keeps the full string for the annotation and as such can be used to generate new STRING format annotations.

Better string representations of unions

If a ForwardRef has to represent a union with a forward reference, this can lead to internal names showing up in the repr and in any future attempt to resolve annotations as strings.

Deferred annotations don't suffer from this issue and will also clean up the names from forward references if they need to be constructed from a ForwardRef.

from annotationlib import get_annotations, Format
from reannotate import get_deferred_annotations, DeferredAnnotation

class Example:
    a: ref | str | bytes

a_anno = get_annotations(Example, format=Format.FORWARDREF)['a']
print(f"{a_anno = }")
print(f"Evaluated as string: {a_anno.evaluate(format=Format.STRING)}")
print()

a_deferred = get_deferred_annotations(Example)['a']
print(f"{a_deferred = }")
print(f"Evaluated as string: {a_deferred.evaluate(format=Format.STRING)}")
print()

# Create a deferred annotation from the ForwardRef
deferred_from_ref = DeferredAnnotation(a_anno)
print(f"{deferred_from_ref = }")
print(f"Evaluated as string: {deferred_from_ref.evaluate(format=Format.STRING)}")

a_anno = ForwardRef('ref | __annotationlib_name_1__ | __annotationlib_name_2__', is_class=True, owner=<class '__main__.Example'>)
Evaluated as string: ref | __annotationlib_name_1__ | __annotationlib_name_2__

a_deferred = DeferredAnnotation('ref | str | bytes')
Evaluated as string: ref | str | bytes

deferred_from_ref = DeferredAnnotation('ref | str | bytes')
Evaluated as string: ref | str | bytes

Use case examples

A 'type' attribute on dataclass-like fields that evaluates

With Python 3.14 annotations, dataclasses can now accept forward references without needing to use __future__ annotations.

Take for example a self referential class:

from dataclasses import dataclass, fields

@dataclass
class Example:
    examples: list[Example]

While this now works, the dataclass 'field' for 'examples' is fixed with the forward reference contained in the 'type' attribute.

examples_field = fields(Example)[0]
print(examples_field.type)

Output:

list[ForwardRef('Example', is_class=True, owner=<class '__main__.Example'>)]

Using reannotate, this can be avoided. Here is the same example but using ducktools-classbuilder instead of dataclasses:

from ducktools.classbuilder.prefab import get_attributes, prefab

@prefab
class Example:
    examples: list[Example]

examples_attribute = get_attributes(Example)['examples']
print(examples_attribute.type)

Output:

list[__main__.Example]

This is because internally, ducktools-classbuilder uses reannotate's get_deferred_annotations instead of Format.FORWARDREF and evaluates them only when .type is accessed.

Adding fields automatically to a dataclass

With the new annotations in Python 3.14 it is no longer always possible to retrieve __annotations__. To correctly handle inserting a field into a dataclass it is necessary to create a new __annotate__ function.

Using get_deferred_annotations and ReAnnotate, this can now be done in a similar fashion as it was possible prior to Python 3.14.

from annotationlib import get_annotations, Format
from dataclasses import dataclass, field
from functools import wraps

from reannotate import get_deferred_annotations, ReAnnotate

def debug_dataclass(cls):
    # Gets all annotations in an unevaluated format
    annos = get_deferred_annotations(cls)

    # Standard objects can be provided and will be converted to `DeferredAnnotation` values
    annos |= {"_used_kwargs": dict[str, object]}

    # ReAnnotate instances are callables that replace the `__annotate__` function
    cls.__annotate__ = ReAnnotate(annos)
    cls._used_kwargs = field(init=False, repr=False, compare=False)

    new_cls = dataclass(cls, slots=True)
    dc_init = new_cls.__init__

    @wraps(dc_init)
    def new_init(self, *args, **kwargs):
        dc_init(self, *args, **kwargs)
        self._used_kwargs = kwargs

    new_cls.__init__ = new_init

    return new_cls

@debug_dataclass
class Example:
    answer: int = 42
    name: str = "Zaphod"
    mystery: Unknown = field(default=None, repr=False)

print(Example()._used_kwargs)  # {}
print(Example(54, name="Dent")._used_kwargs)  # {'name': 'Dent'}

# Define Unknown here and it will allow the annotations to evaluate
Unknown = None | str
print(get_annotations(Example))
# {'answer': <class 'int'>, 'name': <class 'str'>, 'mystery': None | str, '_used_kwargs': dict[str, object]}

Checking which annotations can be evaluated

With the FORWARDREF format, it is not simple to know which annotations would fail to evaluate as forward references can be contained in other arbitrary objects.

DeferredAnnotation instances have an .is_resolved property which indicates if the annotation has been fully evaluated.

from annotationlib import Format
from reannotate import get_deferred_annotations

def f(a: str, b: list[undefined]): ...

annos = get_deferred_annotations(f)

print(annos['a'].evaluate(format=Format.FORWARDREF))  # <class 'str'>
print(annos['a'].is_resolved)  # True
print(annos['b'].evaluate(format=Format.FORWARDREF))  # list[ForwardRef('undefined', ...)]
print(annos['b'].is_resolved)  # False

What about...

Metaclasses

call_annotate_deferred is provided to retrieve deferred annotations in the same way that call_annotate_function is used to retrieve standard annotations.

__future__ annotations

Deferred annotations are intended to act like regular annotations when called with the standard annotation evaluation methods in order to create new __annotate__ functions that behave like the original.

If __future__ annotations are used, get_deferred_annotations will still get DeferredAnnotation objects, but all formats will evaluate to strings, as they do for __future__ annotations with annotationlib.get_annotations.

Literal string annotations

Literal strings in annotations are treated as if they are from __future__ annotations. They will not have an associated evaluation context to prevent accidental attempts at evaluation. This is done to be consistent with how they would be returned from get_annotations without eval_str.

Type Aliases

Like get_annotations, type aliases inside DeferredAnnotation objects will not be evaluated.

from reannotate import get_deferred_annotations

type Vector = list[float]

def f(v: Vector): ...

v_anno = get_deferred_annotations(f)['v']
print(v_anno.evaluate())  # Vector

What about getting this in the stdlib?

Ideally I would like to get this kind of functionality from the stdlib, as currently it relies on a number of private functions from annotationlib. This started as a fork of annotationlib to add deferred annotations as a Format supported directly.

As part of changing this to a third party module, the Format enum value was dropped and make_annotate_function created __annotate__ functions are replaced with the ReAnnotate callable in order to support retrieving deferred annotations from generated annotate callables.

You can read this discourse thread for the origins of this.