pytest-revealtype-injector 0.9.0

Pytest plugin for replacing reveal_type() calls inside test functions with static and runtime type checking result comparison, for confirming type annotation validity.

pytest-revealtype-injector is a pytest plugin for replacing reveal_type() calls inside test functions as something more sophisticated. It does the following tasks in parallel:

• Launch external static type checkers and store reveal_type results.
• Use typeguard to verify the aforementioned static type checker result really matches runtime code result.

Usage

TL;DR:

1. Install this plugin
2. Install type checkers: basedpyright, mypy, pyrefly, pyright, ty
   • Disable any unwanted with --revealtype-disable-adapter=<ADAPTER> pytest CLI option
3. Create pytest functions which call reveal_type() with variable or function return result

The longer story

This plugin would be automatically enabled when launching pytest.

For using reveal_type() inside tests, there is no boiler plate code involved. Import reveal_type normally, like:

from typing import reveal_type

If you care about compatibility with older pythons, use:

import sys
if sys.version >= (3, 11):
    from typing import reveal_type
else:
    from typing_extensions import reveal_type

Just importing typing (or typing_extensions) module is fine too:

import typing

def test_something():
    x: str = 1  # type: ignore  # pyright: ignore
    typing.reveal_type(x)  # typeguard fails here

Since this plugin scans for reveal_type() for replacement under carpet, even import ... as ... syntax works:

import typing as typ  # or...
from typing import reveal_type as rt

To supply config file specific for certain type checker, use --revealtype-<ADAPTER>-config=<FILE> pytest CLI option. For example, --revealtype-pyrefly-config=tests/pyrefly.toml instructs pyrefly to use pyrefly.toml under tests folder to override project root config.

Limitations

There are 3 caveats.

1. This plugin only searches for global import in test files, so local import inside test function doesn't work. That means following code doesn't utilize this plugin at all:

def test_something():
    from typing import reveal_type
    x = 1
    reveal_type(x)  # calls vanilla reveal_type()

2. reveal_type() calls have to stay within a single line, although you can use reveal_type result in assertion or other purpose:

x = "1"
assert reveal_type(str(int(x))) == x

3. This plugin does not enlist any type checker as dependency, because any of them can be optionally disabled with pytest marker (see below) or command line option. It is up to application or library authors to include suitable type checker(s) as dependency themselves.

Disable type checker with marker

Using pytest marker, it is possible to disable usage of certain type checker for specific test. All 3 types of markers (function, class and module level) are supported.

Function level:

@pytest.mark.notypechecker("mypy")
def test_something(self) -> None:
    x = 1
    reveal_type(x)

Class level:

@pytest.mark.notypechecker("pyright")
class TestSomething:
    def test_foo(self) -> None:
    ...

Module level:

pytestmark = pytest.mark.notypechecker("basedpyright", "pyright")

Conversely, it is possible to only turn on usage of specific type checkers with onlytypechecker marker and exclude all others:

@pytest.mark.onlytypechecker("mypy")
def test_for_mypy() -> None:
    ......

Note that disabling all type checkers is disallowed, and such tests would be treated as failure. Disable the reveal_type() call instead.

Logging

This plugin uses standard logging internally. pytest -v can be used to reveal INFO and DEBUG logs. Given following example:

def test_superfluous(self) -> None:
    x: list[str] = ['a', 'b', 'c', 1]  # type: ignore  # pyright: ignore
    reveal_type(x)

Something like this will be shown as test result:

...
    raise TypeCheckError(f"is not an instance of {qualified_name(origin_type)}")
E   typeguard.TypeCheckError: item 3 is not an instance of str (from pyright)
------------------------------------------------------------- Captured log call -------------------------------------------------------------
INFO     revealtype-injector:hooks.py:26 Replaced reveal_type() from global import with <function revealtype_injector at 0x00000238DB923D00>
DEBUG    revealtype-injector:main.py:60 Extraction OK: code='reveal_type(x)', result='x'
========================================================== short test summary info ==========================================================
FAILED tests/runtime/test_attrib.py::TestAttrib::test_superfluous - typeguard.TypeCheckError: item 3 is not an instance of str (from pyright)
============================================================= 1 failed in 3.38s =============================================================

History

This pytest plugin starts its life as part of testsuite related utilities within types-lxml. As lxml is a cython project and probably never incorporate inline python annotation in future, there is need to compare runtime result to static type checker output for discrepancy. As time goes by, it starts to make sense to manage as an independent project.

License-wise, originally it was part of types-lxml project, which is released under Apache-2.0 license. But as the sole author, it is at my own discretion to follow pytest license (which is MIT) because this project is taking shape as a pytest plugin.