Ena Škopelja
03/28/2023, 12:05 PMStructuredDataset
that's failing if I turn on cache_serialize
with this error:
[3/3] currentAttempt done. Last Error: SYSTEM::Traceback (most recent call last):
File "/opt/venv/lib/python3.9/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
return wrapped(*args, **kwargs)
File "/opt/venv/lib/python3.9/site-packages/flytekit/core/base_task.py", line 572, in dispatch_execute
raise TypeError(
Failed to convert return value for var o0 for function {my task} with error <class 'pyarrow.lib.ArrowInvalid'>: ("Could not convert ('... ... (580 characters truncated) ... ...',) with type Row: did not recognize Python value type when inferring an Arrow data type", 'Conversion failed for column sequence with type object')
The sequence column of the dataframe I return is a string (no non-standard characters).
Any idea on what could be causing this? Regular (no cache_serialize
) cache works just fine.Ketan (kumare3)
Ena Škopelja
03/29/2023, 8:47 AMKetan (kumare3)