optype 0.4.0

Building blocks for precise & flexible type hints

optype

One protocol, one method.

Building blocks for precise & flexible type hints.

---

Installation

Optype is available as optype on PyPI:

pip install optype

Example

Let's say you're writing a twice(x) function, that evaluates 2 * x. Implementing it is trivial, but what about the type annotations?

Because twice(2) == 4, twice(3.14) == 6.28 and twice('I') = 'II', it might seem like a good idea to type it as twice[T](x: T) -> T: .... However, that wouldn't include cases such as twice(True) == 2 or twice((42, True)) == (42, True, 42, True), where the input- and output types differ. Moreover, twice should accept any type with a custom __rmul__ method that accepts 2 as argument.

This is where optype comes in handy, which has single-method protocols for all the builtin special methods. For twice, we can use optype.CanRMul[X, Y], which, as the name suggests, is a protocol with (only) the def __rmul__(self, x: X) -> Y: ... method. With this, the twice function can written as:

Python 3.11	Python 3.12
from typing import Literal, TypeAlias, TypeVar from optype import CanRMul Y = TypeVar('Y') Two: TypeAlias = Literal[2] def twice(x: CanRMul[Two, Y]) -> Y: return 2 * x	from typing import Literal from optype import CanRMul type Two = Literal[2] def twice[Y](x: CanRMul[Two, Y]) -> Y: return 2 * x

But what about types that implement __add__ but not __radd__? In this case, we could return x * 2 as fallback (assuming commutativity). Because the optype.Can* protocols are runtime-checkable, the revised twice2 function can be compactly written as:

Python 3.11	Python 3.12
from optype import CanMul def twice2(x: CanRMul[Two, Y] | CanMul[Two, Y]) -> Y: return 2 * x if isinstance(x, CanRMul) else x * 2	from optype import CanMul def twice2[Y](x: CanRMul[Two, Y] | CanMul[Two, Y]) -> Y: return 2 * x if isinstance(x, CanRMul) else x * 2

See examples/twice.py for the full example.

Overview

The API of optype is flat; a single import optype is all you need. There are four flavors of things that live within optype,

• optype.Can{} types describe what can be done with it. For instance, any CanAbs[T] type can be used as argument to the abs() builtin function with return type T. Most Can{} implement a single special method, whose name directly matched that of the type. CanAbs implements __abs__, CanAdd implements __add__, etc.
• optype.Has{} is the analogue of Can{}, but for special attributes. HasName has the __name__: str attribute, HasDict has a __dict__, etc.
• optype.Does{} describe the type of operators. So DoesAbs is the type of the abs({}) builtin function, and DoesPos the type of the +{} prefix operator.
• optype.do_{} are the correctly-typed implementations of Does{}. For each do_{} there is a Does{}, and vice-versa. So do_abs: DoesAbs is the typed alias of abs({}), and do_pos: DoesPos is a typed version of operator.pos. The optype.do_ operators are more complete than operators, has runtime-accessible type annotations, and uses a fully predictable naming scheme.

Reference

All typing protocols here live in the root optype namespace. They are runtime-checkable so that you can do e.g. isinstance('snail', optype.CanAdd), in case you want to check whether snail implements __add__.

[!NOTE] It is bad practice to use a typing.Protocol as base class for your implementation. Because of @typing.runtime_checkable, you can use isinstance either way.

Unlikecollections.abc, optype's protocols aren't abstract base classes, i.e. they don't extend abc.ABC, only typing.Protocol. This allows the optype protocols to be used as building blocks for .pyi type stubs.

Type conversion

The return type of these special methods is invariant. Python will raise an error if some other (sub)type is returned. This is why these optype interfaces don't accept generic type arguments.

operator	operand
expression	function	type	method	type
bool(_)	do_bool	DoesBool	__bool__	CanBool
int(_)	do_int	DoesInt	__int__	CanInt
float(_)	do_float	DoesFloat	__float__	CanFloat
complex(_)	do_complex	DoesComplex	__complex__	CanComplex
bytes(_)	do_bytes	DoesBytes	__bytes__	CanBytes
str(_)	do_str	DoesStr	__str__	CanStr

These formatting methods are allowed to return instances that are a subtype of the str builtin. The same holds for the __format__ argument. So if you're a 10x developer that wants to hack Python's f-strings, but only if your type hints are spot-on; optype is you friend.

operator	operand
expression	function	type	method	type
repr(_)	do_repr	DoesRepr	__repr__	CanRepr[Y: str]
format(_, x)	do_format	DoesFormat	__format__	CanFormat[X: str, Y: str]

"Rich comparison" operators

These special methods generally a bool. However, instances of any type can be returned.

operator	operand
expression	reflected	function	type	method	type
_ < x	x > _	do_lt	DoesLt	__lt__	CanLt[X, Y]
_ <= x	x >= _	do_le	DoesLe	__le__	CanLe[X, Y]
_ == x	x == _	do_eq	DoesEq	__eq__	CanEq[X, Y]
_ != x	x != _	do_ne	DoesNe	__ne__	CanNe[X, Y]
_ > x	x < _	do_gt	DoesGt	__gt__	CanGt[X, Y]
_ >= x	x <= _	do_ge	DoesGe	__ge__	CanGe[X, Y]

Callable objects

Unlike operator, optype provides the operator for callable objects: optype.do_call(f, *args. **kwargs).

CanCall is similar to collections.abc.Callable, but is runtime-checkable, and doesn't use esoteric hacks.

operator	operand
expression	function	type	method	type
_(*args, **kwargs)	do_call	DoesCall	__call__	CanCall[**Xs, Y]

Numeric operations

For describing things that act like numbers. See the Python docs for more info.

operator	operand
expression	function	type	method	type
_ + x	do_add	DoesAdd	__add__	CanAdd[X, Y]
_ - x	do_sub	DoesSub	__sub__	CanSub[X, Y]
_ * x	do_mul	DoesMul	__mul__	CanMul[X, Y]
_ @ x	do_matmul	DoesMatmul	__matmul__	CanMatmul[X, Y]
_ / x	do_truediv	DoesTruediv	__truediv__	CanTruediv[X, Y]
_ // x	do_floordiv	DoesFloordiv	__floordiv__	CanFloordiv[X, Y]
_ % x	do_mod	DoesMod	__mod__	CanMod[X, Y]
divmod(_, x)	do_divmod	DoesDivmod	__divmod__	CanDivmod[X, Y]
_ ** x pow(_, x)	do_pow/2	DoesPow	__pow__	CanPow2[X, Y2] CanPow[X, None, Y2, Any]
pow(_, x, m)	do_pow/3	DoesPow	__pow__	CanPow3[X, M, Y3] CanPow[X, M, Any, Y3]
_ << x	do_lshift	DoesLshift	__lshift__	CanLshift[X, Y]
_ >> x	do_rshift	DoesRshift	__rshift__	CanRshift[X, Y]
_ & x	do_and	DoesAnd	__and__	CanAnd[X, Y]
_ ^ x	do_xor	DoesXor	__xor__	CanXor[X, Y]
_ | x	do_or	DoesOr	__or__	CanOr[X, Y]

Note that because pow() can take an optional third argument, optype provides separate interfaces for pow() with two and three arguments. Additionally, there is the overloaded intersection type CanPow[X, M, Y2, Y3] =: CanPow2[X, Y2] & CanPow3[X, M, Y3], as interface for types that can take an optional third argument.

For the binary infix operators above, optype additionally provides interfaces with reflected (swapped) operands:

operator	operand
expression	function	type	method	type
x + _	do_radd	DoesRAdd	__radd__	CanRAdd[X, Y]
x - _	do_rsub	DoesRSub	__rsub__	CanRSub[X, Y]
x * _	do_rmul	DoesRMul	__rmul__	CanRMul[X, Y]
x @ _	do_rmatmul	DoesRMatmul	__rmatmul__	CanRMatmul[X, Y]
x / _	do_rtruediv	DoesRTruediv	__rtruediv__	CanRTruediv[X, Y]
x // _	do_rfloordiv	DoesRFloordiv	__rfloordiv__	CanRFloordiv[X, Y]
x % _	do_rmod	DoesRMod	__rmod__	CanRMod[X, Y]
divmod(x, _)	do_rdivmod	DoesRDivmod	__rdivmod__	CanRDivmod[X, Y]
x ** _ pow(x, _)	do_rpow	DoesRPow	__rpow__	CanRPow[X, Y]
x << _	do_rlshift	DoesRLshift	__rlshift__	CanRLshift[X, Y]
x >> _	do_rrshift	DoesRRshift	__rrshift__	CanRRshift[X, Y]
x & _	do_rand	DoesRAnd	__rand__	CanRAnd[X, Y]
x ^ _	do_rxor	DoesRXor	__rxor__	CanRXor[X, Y]
x | _	do_ror	DoesROr	__ror__	CanROr[X, Y]

Note that CanRPow corresponds to CanPow2; the 3-parameter "modulo" pow does not reflect in Python.

Similarly, the augmented assignment operators are described by the following optype interfaces:

operator	operand
expression	function	type	method	type
_ += x	do_iadd	DoesIAdd	__iadd__	CanIAdd[X, Y]
_ -= x	do_isub	DoesISub	__isub__	CanISub[X, Y]
_ *= x	do_imul	DoesIMul	__imul__	CanIMul[X, Y]
_ @= x	do_imatmul	DoesIMatmul	__imatmul__	CanIMatmul[X, Y]
_ /= x	do_itruediv	DoesITruediv	__itruediv__	CanITruediv[X, Y]
_ //= x	do_ifloordiv	DoesIFloordiv	__ifloordiv__	CanIFloordiv[X, Y]
_ %= x	do_imod	DoesIMod	__imod__	CanIMod[X, Y]
_ **= x	do_ipow	DoesIPow	__ipow__	CanIPow[X, Y]
_ <<= x	do_ilshift	DoesILshift	__ilshift__	CanILshift[X, Y]
_ >>= x	do_irshift	DoesIRshift	__irshift__	CanIRshift[X, Y]
_ &= x	do_iand	DoesIAnd	__iand__	CanIAnd[X, Y]
_ ^= x	do_ixor	DoesIXor	__ixor__	CanIXor[X, Y]
_ |= x	do_ior	DoesIOr	__ior__	CanIOr[X, Y]

Additionally, there are the unary arithmetic operators:

operator	operand
expression	function	type	method	type
+_	do_pos	DoesPos	__pos__	CanPos[Y]
-_	do_neg	DoesNeg	__neg__	CanNeg[Y]
~_	do_invert	DoesInvert	__invert__	CanInvert[Y]
abs(_)	do_abs	DoesAbs	__abs__	CanAbs[Y]

The round() built-in function takes an optional second argument. From a typing perspective, round() has two overloads, one with 1 parameter, and one with two. For both overloads, optype provides separate operand interfaces: CanRound1[Y] and CanRound2[N, Y]. Additionally, optype also provides their (overloaded) intersection type: CanRound[N, Y1, Y2] = CanRound1[Y1] & CanRound2[N, Y2].

operator	operand
expression	function	type	method	type
round(_)	do_round/1	DoesRound	__round__/1	CanRound1[Y1] CanRound[None, Y1, Any]
round(_, n)	do_round/2	DoesRound	__round__/2	CanRound2[N, Y2] CanRound[N, Any, Y2]
round(_, n=...)	do_round/1 do_round/2	DoesRound	__round__	CanRound[N, Y1, Y2]

For example, type-checkers will mark the following code as valid (tested with pyright in strict mode):

x: float = 3.14
x1: CanRound1[int] = x
x2: CanRound2[int, float] = x
x3: CanRound[int, int, float] = x

Furthermore, there are the alternative rounding functions from the math standard library:

operator	operand
expression	function	type	method	type
math.trunc(_)	do_trunc	DoesTrunc	__trunc__	CanTrunc[Y]
math.floor(_)	do_floor	DoesFloor	__floor__	CanFloor[Y]
math.ceil(_)	do_ceil	DoesCeil	__ceil__	CanCeil[Y]

Note that the type parameter Y has no upper type bound, because technically these methods can return any type. However, in practise, it is very common to have them return an int.

Async objects

The optype variant of collections.abc.Awaitable[V]. The only difference is that optype.CanAwait[V] is a pure interface, whereas Awaitable is also an abstract base class.

operator	operand
expression	method	type
await _	__await__	CanAwait[V]

Iteration

The operand x of iter(_) is within Python known as an iterable, which is what collections.abc.Iterable[K] is often used for (e.g. as base class, or for instance checking).

The optype analogue is CanIter[Ks], which as the name suggests, also implements __iter__. But unlike Iterable[K], its type parameter Ks binds to the return type of iter(_). This makes it possible to annotate the specific type of the iterable that iter(_) returns. Iterable[K] is only able to annotate the type of the iterated value. To see why that isn't possible, see python/typing#548.

The collections.abc.Iterator[K] is even more awkward; it is a subtype of Iterable[K]. For those familiar with collections.abc this might come as a surprise, but an iterator only needs to implement __next__, __iter__ isn't needed. This means that the Iterator[K] is unnecessarily restrictive. Apart from that being theoretically "ugly", it has significant performance implications, because the time-complexity of isinstance on a typing.Protocol is $O(n)$, with the $n$ referring to the amount of members. So even if the overhead of the inheritance and the abc.ABC usage is ignored, collections.abc.Iterator is twice as slow as it needs to be.

That's one of the (many) reasons that optype.CanNext[V] and optype.CanNext[V] are the better alternatives to Iterable and Iterator from the abracadabra collections. This is how they are defined:

operator	operand
expression	function	type	method	type
next(_)	do_next/1	DoesNext	__next__	CanNext[V]
iter(_)	do_iter/1	DoesIter	__iter__	CanIter[Vs: CanNext]

For the sake of compatibility with collections.abc, there is optype.CanIterSelf[T], which is a protocol whose __iter__ returns typing.Self, as well as a __next__ method that returns T. I.e. it is equivalent to collections.abc.Iterator[T], but without the abc nonsense.

Async Iteration

Yes, you guessed it right; the abracadabra collections made the exact same mistakes for the async iterablors (or was it "iteramblers"...?).

But fret not; the optype alternatives are right here:

operator	operand
expression	function	type	method	type
anext(_)	do_anext	DoesANext	__anext__	CanANext[V]
aiter(_)	do_aiter	DoesAIter	__aiter__	CanAIter[Vs: CanAnext]

But wait, shouldn't V be a CanAwait? Well, only if you don't want to get fired... Technically speaking, __anext__ can return any type, and anext will pass it along without nagging (instance checks are slow, now stop bothering that liberal). For details, see the discussion at python/typeshed#7491. Just because something is legal, doesn't mean it's a good idea (don't eat the yellow snow).

Additionally, there is optype.CanAIterSelf[V], with both the __aiter__() -> Self and the __anext__() -> V methods.

Containers

operator	operand
expression	function	type	method	type
len(_)	do_len	DoesLen	__len__	CanLen
_.__length_hint__() (docs)	do_length_hint	DoesLengthHint	__length_hint__	CanLengthHint
_[k]	do_getitem	DoesGetitem	__getitem__	CanGetitem[K, V]
_.__missing__() (docs)	do_missing	DoesMissing	__missing__	CanMissing[K, V]
_[k] = v	do_setitem	DoesSetitem	__setitem__	CanSetitem[K, V]
del _[k]	do_delitem	DoesDelitem	__delitem__	CanDelitem[K]
k in _	do_contains	DoesContains	__contains__	CanContains[K]
reversed(_)	do_reversed	DoesReversed	__reversed__	CanReversed[Vs] | CanSequence[V]

Because CanMissing[K, M] generally doesn't show itself without CanGetitem[K, V] there to hold its hand, optype conveniently stitched them together as optype.CanGetMissing[K, V, M].

Similarly, there is optype.CanSequence[I: CanIndex, V], which is the combination of both CanLen and CanItem[I, V], and serves as a more specific and flexible collections.abc.Sequence[V].

Additionally, optype provides protocols for types with (custom) hash or index methods:

operator	operand
expression	function	type	method	type
hash(_)	do_hash	DoesHash	__hash__	CanHash[V]
_.__index__() (docs)	do_index	DoesIndex	__index__	CanIndex[V]

Attribute access

operator	operand
expression	function	type	method	type
v = _.k or v = getattr(_, k)	do_getattr	DoesGetattr	__getattr__	CanGetattr[K: str, V]
_.k = v or setattr(_, k, v)	do_setattr	DoesSetattr	__setattr__	CanSetattr[K: str, V]
del _.k or delattr(_, k)	do_delattr	DoesDelattr	__delattr__	CanDelattr[K: str]
dir(_)	do_dir	DoesDir	__dir__	CanDir[Vs: CanIter]

Descriptors

Interfaces for descriptors.

operator	operand
expression	method	type
class T: d = _	__set_name__	CanSetName[T]
u = T.d v = T().d	__get__	CanGet[T: object, U, V]
T().k = v	__set__	CanSet[T, V]
del T().k	__delete__	CanDelete[T]

Context managers

Support for the with statement.

operator	operand
expression	method(s)	type
with _ as v	__enter__, __exit__	CanWith[V, R]
	__enter__	CanEnter[V]
	__exit__	CanExit[R]

For the async with statement the interfaces look very similar:

operator	operand
expression	method(s)	type
async with _ as v	__aenter__, __aexit__	CanAsyncWith[V, R]
	__aenter__	CanAEnter[V]
	__aexit__	CanAExit[R]

Buffer types

Interfaces for emulating buffer types using the buffer protocol.

operator	operand
expression	method	type
v = memoryview(_)	__buffer__	CanBuffer[B: int]
del v	__release_buffer__	CanReleaseBuffer

copy

For the copy standard library, optype provides the following interfaces:

operator	operand
expression	method	type
copy.copy(_)	__copy__	CanCopy[T]
copy.deepcopy(_, memo={})	__deepcopy__	CanDeepcopy[T]
copy.replace(_, **changes: V) (Python 3.13+)	__replace__	CanReplace[T, V]

And for convenience, there are the runtime-checkable aliases for all three interfaces, with T bound to Self. These are roughly equivalent to:

type CanCopySelf = CanCopy[CanCopySelf]
type CanDeepcopySelf = CanDeepcopy[CanDeepcopySelf]
type CanReplaceSelf[V] = CanReplace[CanReplaceSelf[V], V]

pickle

For the pickle standard library, optype provides the following interfaces:

method(s)	signature (bound)	type
__reduce__	() -> R	CanReduce[R: str | tuple]
__reduce_ex__	(CanIndex) -> R	CanReduceEx[R: str | tuple]
__getstate__	() -> State	CanGetstate[State: object]
__setstate__	(State) -> None	CanSetstate[State: object]
__getnewargs__ __new__	() -> tuple[*Args] (*Args) -> Self	CanGetnewargs[*Args]
__getnewargs_ex__ __new__	() -> tuple[tuple[*Args], dict[str, Kw]] (*Args, **dict[str, Kw]) -> Self	CanGetnewargsEx[*Args, Kw]

dataclasses

For the dataclasses standard library, optype provides the optype.HasDataclassFields interface It can conveniently be used to check whether a type or instance is a dataclass, i.e. isinstance(obj, optype.HasDataclassFields).

Future plans

• numpy interfaces for arrays-like types (no deps) (#24)
• array-api interfaces (no deps) (#25)
• dataframe-api interfaces (no deps) (#26)