https://flyte.org logo
#ask-the-community
Title
# ask-the-community
g

Gopal Vashishtha

10/13/2023, 6:37 PM
What are possible causes of
Copy code
RuntimeExecutionError: max number of system retry attempts [51/50] exhausted. Last known status message: failed at Node[n2]. EventRecordingFailed: failed to record node event, caused by: EventSinkError: Error sending event, caused by [rpc error: code = DeadlineExceeded desc = context deadline exceeded]
when running a dynamic task that iterates over a list of 35K inputs and spins up map tasks on batches of 500 inputs? Checked flyte-propeller logs in CloudWatch, didn't see anything informative
k

Ketan (kumare3)

10/14/2023, 7:27 PM
I think FlyteAdmin is timing out writing some event log
this informs when a node in the workflow is completed etc
It is your db falling over most likely
Cc @Dan Rammer (hamersaw) / @Haytham Abuelfutuh fyi
3 Views