Hey community, I am facing an issue, which is the following: RuntimeExecutionError: failed during plugin execution, caused by: output file @[s3://flyte/metadata/propeller/bap-development-apmzcrx7jtppgfdx5qfk/n7/data/0/n0/0/outputs.pb] is too large [142225647] bytes, max allowed [10485760] bytes ,where my expected output type is

List[AUResultsWrapperPerSegment]

It seems like the "descriptor" protobuf file has become too large, which I don't understand why. I have checked and I have a 29 long list with my custom dataclass as output:

@dataclass_json
@dataclass
class AUResultsWrapperPerSegment:
    segment_id: int
    preds: Dict[str, List[float]]

Can you tell me what to do otherwise? I don't want to change the limit since the max allowed [10485760] bytes is already large.

You can use files For example a list of dataclass and instead of dictionary use a dataframe?

Daraframe could work, but why does it result in a large output protobuf file with a dictionary inside a dataclass?

It’s serializing all of the data to proto

I understand, thanks for the help!