Flyte Pandera Plugin

flyteplugins-pandera adds support for pandera.typing.pandas.DataFrame, pandera.typing.polars.DataFrame / LazyFrame, and pandera.typing.pyspark_sql.DataFrame in Flyte v2.

Install:

pip install flyteplugins-pandera 'pandera[pandas]'            # pandas only
pip install flyteplugins-pandera 'pandera[polars]' flyteplugins-polars   # Polars + structured dataset I/O
pip install flyteplugins-pandera 'pandera[pyspark]' flyteplugins-spark     # PySpark SQL + parquet I/O

For PySpark, structured dataset serialization uses Flyte’s DataFrameTransformerEngine parquet handlers from flyteplugins-spark (register Spark encoders/decoders alongside this plugin).

At runtime, the plugin:

delegates dataframe IO to Flyte's DataFrameTransformerEngine,
validates data with pandera schemas, and
writes a validation report to flyte.report.

Validation always runs on every encode/decode. Report tabs are suppressed automatically when Flyte is only moving literals across a nested-task boundary (parent task encoding child inputs, or materializing a child’s outputs inside the parent). The SDK sets TaskContext.in_driver_literal_conversion on the active task (check with flyte.ctx() when non-None, then .in_driver_literal_conversion) so you see one report per dataframe on the task that actually produced or consumed it as task body I/O, not extra tabs on the orchestrating “driver” task.

Troubleshooting

If logs show “Unsupported Type pandera.typing… Flyte will default to use PickleFile”, the pandera transformer was not registered:

Install the plugin in every environment (local runner and task image): pip install flyteplugins-pandera.
Flyte loads flyte.plugins.types during flyte.initialize() and on first TypeEngine use; confirm the distribution is installed (import importlib.metadata as m; print(list(m.entry_points(group="flyte.plugins.types")))).
Import order: import your pandera.typing.* modules before plugin registration runs in files that run early (tests, __init__.py). Loading the plugin before pandera can leave two different pandera.typing.pandas.DataFrame (or polars container) class objects in the process; TypeEngine would only know about one of them, so annotations on the other fall through to pickle / the generic handler.

Release files for flyteplugins-pandera 2.10.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for flyteplugins-pandera 2.10.1
File	Interpreter	ABI	Platform	Reset
flyteplugins_pandera-2.10.1-py3-none-any.whl 17.0 kB Sep 25, 2026	Python 3	none	any	Details

Release files / flyteplugins_pandera-2.10.1-py3-none-any.whl

Download URL	flyteplugins_pandera-2.10.1-py3-none-any.whl
Size	17.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`9ba00af52764c8d361622707d2114b351b0cda271abf352222c8d4f559f7e3fe`
BLAKE2b-256 checksum How to use checksums	`630dce1d1a967d21d49bf4df0d832519cd1ec4db07eb2e59e760fb6fd1b0a6b6`
Upload date	Sep 25, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.13.15`