https://flyte.org logo
Join Slack
Powered by
Hi I have a task that expects an input of custom p...
# ask-the-community
n
Hi I have a task that expects an input of custom pydantic model - lets say
Copy code
from pydantic import BaseModel

class CustomInput(BaseModel):
    field1: ...
    field2: ...

@task
def do_something(config: CustomInput):
    ....
previously, I used dataclass instead of pydantic BaseModel and on the UI it accepted json object as input. However, now with BaseModel, a warning shows up on registration/running saying that this type is not supported by default and will default to a pythonPickle.. and on the UI it only accepts a pickle file now .
Copy code
Unsupported Type <class 'CustomInput'> found, Flyte will default to use PickleFile as the transport. Pickle can only be used to send objects between the exact same version of Python, and we strongly recommend to use python type that flyte support.
Later on, If a pickle file is passed, it doesn't get converted to CustomInput object. I installed pydantic plugin flytekit 1.11.0 flytekitplugins-pydantic 1.11.0 pydantic 2.7.0 Am I doing something wrong or is this the expected behaviour?
g
I use 
@pydantic.dataclasses.dataclass
and inherit from 
DataClassJSONMixin
It works quite well.
n
but does it support @computed_field and property?
g
I am unsure. I can run a quick test but I would think so. Regardless, you use 
@cached_property
. My understanding to the difference is that the property of a computed field is cached between multiple components, whereas a @cached_property is only cached within each individual component. So, effectively, you would need to re-compute the property for each new task.
I know that it works with property. Not at my computer yet to test if it works with computed field.
n
I am currently testing it with property, thanks
g
Yeah, I am able to run it with 
@computed_field
and it (de)serializes as expected.
Copy code
from typing import Annotated

import pydantic
from mashumaro.mixins.json import DataClassJSONMixin

@pydantic.dataclasses.dataclass
class Hyperparameters(DataClassJSONMixin):

    batch_size: Annotated[int, pydantic.Field(gt=0, le=2048)] = 128
    n_context: Annotated[int, pydantic.Field(gt=0, le=2048)] = 128
    d_hidden: Annotated[int, pydantic.Field(gt=1, le=256)] = 64

    @pydantic.computed_field
    def memory_complexity(self) -> int:

        return self.n_context * self.n_context * self.d_hidden + self.n_context * self.d_hidden * self.d_hidden
Open in Slack
PreviousNext
Powered by