Hi all, I have a task returning a

StructuredDataset

that's failing if I turn on

cache_serialize

with this error:

[3/3] currentAttempt done. Last Error: SYSTEM::Traceback (most recent call last):

      File "/opt/venv/lib/python3.9/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
        return wrapped(*args, **kwargs)
      File "/opt/venv/lib/python3.9/site-packages/flytekit/core/base_task.py", line 572, in dispatch_execute
        raise TypeError(
    Failed to convert return value for var o0 for function {my task} with error <class 'pyarrow.lib.ArrowInvalid'>: ("Could not convert ('... ... (580 characters truncated) ... ...',) with type Row: did not recognize Python value type when inferring an Arrow data type", 'Conversion failed for column sequence with type object')

The sequence column of the dataframe I return is a string (no non-standard characters). Any idea on what could be causing this? Regular (no

cache_serialize

) cache works just fine.

Imp cache serialize has nothing to do with it. Seems like a data issue. Some value is unexpected

Why does it work with regular cache then? It's a dataclass with string columns

Hmm this is really odd