#66 Translate pydantic to dataframe Pull request opened by wild-endeavor This turns the DataFrame object from a dataclass to a pydantic model, to match the File/Dir classes. Changes: • Update the base class to Pydantic • The class only has two args in the constructor uri and format. The rest of the fields have been marked as private fields and are for internal use only. • Direct instantiation is still allowed but the examples don't show it and it's not used internally either. Instead there's a wrapper function called

create_from

that among other things takes a native dataframe (like pandas) object. • I tried to get rid of the

create_from

and implement a

from_local

similar to the File/Dir experience, and I think we should but it felt a bit trickier so I don't want to do that now. The issue is that while we can do the upload, it's harder to construct the entire literal. The DataFrame type (or any native dataframe type) takes additional arguments in the form of an

Annotated

annotation on the type. This can add column information as well as the storage format, the latter of which can affect the encoder that's used (which in turn affects how the upload happens). I don't think we should we should make the user declare the annotations twice - we could alternatively remove it from the type signature, and put everything in the

from_local

call, but this also doesn't work well when we're only working with types (as in, setting the interface for a task, which includes column and format information). Needs more cleanup following this PR • Add a better example. • Finalize constructor methods (remove metadata proto dependency and remove all usage of raw constructor). • Update tests. flyteorg/flyte-sdk