lively-sundown-82704
08/29/2022, 7:54 PMFlyteSchema
object that also includes metadata on whether a particular column is nullable vs required, along with its type?thankful-minister-83577
thankful-minister-83577
lively-sundown-82704
08/29/2022, 8:10 PMlively-sundown-82704
08/29/2022, 8:14 PMkwtypes
, which upon introspecting the code looks like it is relying on Python's native type system
so a different phrasing of this question is: do Structured Datasets / FlyteSchemas allow for Optional
/ Union
type annotations in their column definitions?famous-businessperson-24711
08/29/2022, 8:55 PMfield: str | None
thankful-minister-83577
thankful-minister-83577
glamorous-carpet-83516
08/30/2022, 8:30 AMcols = kwtypes(Name=str, Age=typing.Optional[int], Height=int)
@task
def get_df(a: int) -> Annotated[pd.DataFrame, cols]:
return pd.DataFrame({"Name": ["Tom", "Joseph"], "Adg": [a, None], "Height": [160, 178]})
glamorous-carpet-83516
08/30/2022, 8:38 AMtyping.Optional
lively-sundown-82704
08/30/2022, 2:08 PMglamorous-carpet-83516
08/30/2022, 2:14 PM