#631 feat: Add Pandera data validation plugin Pull request opened by andreahlert Summary Port the Pandera plugin from flytekit v1 to the Flyte v2 SDK, enabling automatic runtime validation of pandas DataFrames against Pandera schemas as data flows between tasks. This brings Pandera support to the v2 SDK, following the same plugin architecture used by polars, wandb, and other existing plugins. What's included •

PanderaTransformer

: A

TypeTransformer

that wraps the

DataFrameTransformerEngine

to add Pandera schema validation on both

to_literal

(output) and

to_python_value

(input) •

ValidationConfig

: Configurable error handling (

raise

or

warn

) via

typing.Annotated

•

PandasReportRenderer

: HTML report generation using

great_tables

for validation results (compatible with Flyte Decks) • Validation memo: Skips redundant re-validation during local execution where

to_literal

is immediately followed by

to_python_value

• Entry point registration: Registered via

flyte.plugins.types

entry point, loaded automatically by

flyte.init()

Usage import flyte import pandas as pd import pandera as pa from pandera.typing import DataFrame, Series env = flyte.TaskEnvironment(name="my-env") class UserSchema(pa.DataFrameModel): name: Series[str] age: Series[int] = pa.Field(ge=0, le=120) email: Series[str] @env.task async def generate_users() -> DataFrame[UserSchema]: return pd.DataFrame({ "name": ["Alice", "Bob"], "age": [25, 30], "email": ["alice@example.com", "bob@example.com"], }) @env.task async def process_users(df: DataFrame[UserSchema]) -> DataFrame[UserSchema]: df["age"] = df["age"] + 1 return df Test plan • Transformer registration with TypeEngine • Literal type generation for pandera DataFrame types • Schema extraction from DataFrameModel annotations • Validation with valid data (to_literal) • Validation failure with invalid data (raises SchemaErrors) • Warn mode with ValidationConfig(on_error="warn") • Roundtrip encode/decode with valid data • Validation memo prevents duplicate validation • Type assertion checks flyteorg/flyte-sdk