New RunLLM response generated: >>>*# Question* Hi team.. I’m running into the same issue as what’s discussed here. It’s a dynamic task that is used to chunk up and trigger around 8000 map tasks… the input to the dynamic task is a list of inputs (dataclasses) for these map tasks… and each one is just 4 s3 urls.. To get around this I serialized the data classes into a single file and read that in the dynamic task to recreate the inputs to the map tasks.. but still running into this error

TooLarge: Event message exceeds maximum gRPC size limit, caused by [rpc error: code = ResourceExhausted desc = grpc: received message larger than max (6192640 vs. 4194304)]

# Answer The error you're encountering is due to the gRPC message size limit being exceeded when Flyte's propeller sends an event to admin. This is a common issue when dealing with large inputs or outputs in Flyte workflows. According to this discussion, one approach to mitigate this issue is to offload large data and pass references (e.g., S3 URLs) instead of the data itself. Would you like more details on how to implement data offloading or other strategies to handle large data in Flyte? Sources: - https://github.com/flyteorg/flyte/pull/5697 - https://github.com/flyteorg/flyte/issues/381 Slack Link https://.slack.com/archives/CP2HDHKE1/p1729797000.374019 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.