Hi community, I am evaluating flyte for some inter...
# ask-the-community
g
Hi community, I am evaluating flyte for some internal use cases and having some issues setting up a slightly more complex workflow using nested dataclasses. I have made a minimal reproduction here. I just want to confirm that this is expected behaviour and we should look at workarounds 🙂 (full code and error in thread)
Copy code
@dataclass
class GenerateMessageResult:
    message: str


@dataclass
class ModifyMessageResult:
    message: str


@dataclass
class FinalOutput:
    message: ModifyMessageResult


@task
def generate_message(number: int) -> GenerateMessageResult:
    return GenerateMessageResult(
        message=f"The number is {number}",
    )


@task
def modify_message(message: GenerateMessageResult) -> ModifyMessageResult:
    return ModifyMessageResult(
        message=f"Modified: {message.message}",
    )


@workflow
def process_number(number: int) -> FinalOutput:
    message = generate_message(number=number)
    modified_message = modify_message(message=message)
    return FinalOutput(
        message=modified_message,
    )


@dynamic
def process_numbers(numbers: typing.List[int]) -> typing.List[FinalOutput]:
    messages: typing.List[FinalOutput] = []
    for number in numbers:
        message = process_number(number=number)
        messages.append(message)
    return messages


@workflow
def wf() -> typing.List[FinalOutput]:
    return process_numbers(numbers=[1, 2, 3, 4, 5])
Copy code
Detected Root /Users/gus/Documents/code/flyte-evaluation/basescan, using this to create deployable package...
No output path provided, using a temporary directory at /var/folders/by/0h9hg3lx1lsfz_xsqw5m9z740000gn/T/tmpb1_io96g instead
Computed version is _dUfitsIsK6mUzrMCA0BMw
Loading packages ['workflows'] under source root /Users/gus/Documents/code/flyte-evaluation/basescan
Traceback:
  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/core/workflow.py", line 748, in compile
    b, _ = binding_from_python_std(
           ^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/core/promise.py", line 768, in binding_from_python_std
    binding_data = binding_data_from_python_std(ctx, expected_literal_type, t_value, t_value_type, nodes)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/core/promise.py", line 756, in binding_data_from_python_std
    scalar = TypeEngine.to_literal(ctx, t_value, t_value_type or type(t_value), expected_literal_type).scalar
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/core/type_engine.py", line 1175, in to_literal
    lv = transformer.to_literal(ctx, python_val, python_type, expected)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/core/type_engine.py", line 486, in to_literal
    self._serialize_flyte_type(python_val, python_type)

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/core/type_engine.py", line 551, in _serialize_flyte_type
    from flytekit.types.schema.types import FlyteSchema

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/types/schema/__init__.py", line 1, in <module>
    from .types import (

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/types/schema/types.py", line 324, in <module>
    class FlyteSchemaTransformer(TypeTransformer[FlyteSchema]):

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/flytekit/types/schema/types.py", line 340, in FlyteSchemaTransformer
    _np.string_: SchemaType.SchemaColumn.SchemaColumnType.STRING,
    ^^^^^^^^^^^

  File "/Users/gus/Documents/code/flyte-evaluation/.venv/lib/python3.12/site-packages/numpy/__init__.py", line 397, in __getattr__
    raise AttributeError(
actually this doesn't happen in only nested dataclasses. Even non nested ones i get errors. It seems when I am using the result of a task in the result dataclass for a workflow.
the issue seems to be with building a dataclass within a workflow function? is this not supported ?
y
could you switch to the StructuredDataset type?
that replaced the Schema some time ago, but we’ve yet to formally deprecate it.
cc @Eduardo Apolinario (eapolinario)