acoustic-carpenter-78188
08/15/2024, 6:39 PMNone
and can't really tease out what exactly causes it. I have a possible fix below, but don't understand enough about the cause to ensure correctness. The basic reproduction involves having one @task
output a nullable value, and then later using that nullable value as input to another @task
or @workflow
. If the value is None
, it fails to deserialize with the following error:
TypeError: Failed to convert inputs of task 'simple_none_repro.wrap_outer_outputs':
Failed argument 'output': Python value cannot be None, expected <class 'NoneType'>/[Flyte Serialized object: Type: <LiteralType>
Value: <simple: NONE>]
Here's a simple repro file:
# simple_none_repro.py
from flytekit import task, workflow
from dataclasses import dataclass
from mashumaro.mixins.json import DataClassJSONMixin
@dataclass
class OuterWorkflowInput(DataClassJSONMixin):
input: float
@dataclass
class OuterWorkflowOutput(DataClassJSONMixin):
nullable_output: float | None = None
@dataclass
class InnerWorkflowInput(DataClassJSONMixin):
input: float
@dataclass
class InnerWorkflowOutput(DataClassJSONMixin):
nullable_output: float | None = None
@task
def inner_task(input: float) -> float | None:
if input == 0:
return None
return input
@task
def wrap_inner_inputs(input: float) -> InnerWorkflowInput:
return InnerWorkflowInput(input=input)
@task
def wrap_inner_outputs(output: float | None) -> InnerWorkflowOutput:
return InnerWorkflowOutput(nullable_output=output)
@task
def wrap_outer_outputs(output: float | None) -> OuterWorkflowOutput:
return OuterWorkflowOutput(nullable_output=output)
@workflow
def inner_workflow(input: InnerWorkflowInput) -> InnerWorkflowOutput:
return wrap_inner_outputs(
output=inner_task(
input=input.input
)
)
@workflow
def outer_workflow(input: OuterWorkflowInput) -> OuterWorkflowOutput:
inner_outputs = inner_workflow(
input=wrap_inner_inputs(input=input.input)
)
return wrap_outer_outputs(
output=inner_outputs.nullable_output
)
if name == "__main__":
print(outer_workflow(OuterWorkflowInput(input=1))) # works fine
print(outer_workflow(OuterWorkflowInput(input=0))) # fails with the above error
I've been able to "resolve" the issue in a local patch to flytekit
that makes the following change to this line:
if (python_val is None and python_type != NoneType) and expected and expected.union_type is None:
Doing this makes the above file run cleanly:
>> python simple_none_repro.py
OuterWorkflowOutput(nullable_output=1.0)
OuterWorkflowOutput(nullable_output=None)
flyteorg/flyteacoustic-carpenter-78188
08/15/2024, 6:39 PM