https://flyte.org logo
#ask-the-community
Title
# ask-the-community
t

Tom Touati

03/31/2024, 11:55 AM
Hey all, Context - I'm running a workflow which runs a sub-workflow that runs a sequence of a few map tasks. I'm getting this error upon registration, when using any float between 0 and 1 in a map_task's min_success_rate.. if I don't use min_success_rate the registration is successful.. any ideas? Code: SYSTEM:Unknown RPC Failed, with Status: StatusCode.RESOURCE_EXHAUSTED details: received metadata size exceeds hard limit (value length 26937 vs. 16384) Debug string UNKNOWN:Error received from peer {grpc_message:"received metadata size exceeds hard limit (value length 26937 vs. 16384)", grpc_status:8, created_time:"2024-03-31T142647.970151344+03:00"}
image.png
k

Ketan (kumare3)

03/31/2024, 2:29 PM
This is really weird that we fail when we set min ratio. The difference seems rather large. The error IMO is from the Grpc layer. Seems like the generated workflow is becoming a large payload than allowed by Grpc limits. If this is so you can infact increase Grpc limits
t

Tom Touati

03/31/2024, 3:04 PM
How can one do that?
@Ketan (kumare3) Okay, fixed it. At first I did not understand the mechanism of "min_success_ratio". After I did - I configured the rest of the workflow so that instead of expecting input type xyz it will expect type Union[xyz,None], so it will account for failed sub tasks. worked.
k

Ketan (kumare3)

03/31/2024, 5:38 PM
Aaha interesting
5 Views