Varun Kulkarni
08/29/2022, 7:54 PMFlyteSchema
object that also includes metadata on whether a particular column is nullable vs required, along with its type?Yee
Varun Kulkarni
08/29/2022, 8:10 PMkwtypes
, which upon introspecting the code looks like it is relying on Python's native type system
so a different phrasing of this question is: do Structured Datasets / FlyteSchemas allow for Optional
/ Union
type annotations in their column definitions?Dylan Wilder
08/29/2022, 8:55 PMfield: str | None
Yee
Kevin Su
08/30/2022, 8:30 AMcols = kwtypes(Name=str, Age=typing.Optional[int], Height=int)
@task
def get_df(a: int) -> Annotated[pd.DataFrame, cols]:
return pd.DataFrame({"Name": ["Tom", "Joseph"], "Adg": [a, None], "Height": [160, 178]})
typing.Optional
Varun Kulkarni
08/30/2022, 2:08 PMKevin Su
08/30/2022, 2:14 PM