abundant-laptop-47033
10/31/2023, 6:45 PMFailed to update workflow. Error [rpc error: code = ResourceExhausted desc = trying to send message larger than max (2159575 vs. 2097152)]"
Can this max value be adjusted in helm? I'm struggling to find exactly where to change it.
I'm using FlytePropeller v1.1.47. Thanks!freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
abundant-laptop-47033
10/31/2023, 9:00 PMtall-lock-23197
thankful-minister-83577
abundant-laptop-47033
11/01/2023, 3:38 PMaverage-finland-92144
11/01/2023, 4:31 PMabundant-laptop-47033
11/01/2023, 4:35 PMflyte-core
helm chart.
I found an option for configuring the max message size in FlyteAdmin but I can't find a corresponding option for FlytePropeller. I can try adjusting maxLogMessageLength
in the meantime.full-ram-17934
11/01/2023, 9:16 PMabundant-laptop-47033
11/02/2023, 8:50 PMfrom flytekit import workflow, task
@task
def raise_giant_error() -> None:
error_line = "This is a really big error"
n_lines = 100000
raise ValueError("\n".join([error_line] * n_lines))
@workflow
def giant_error_wf() -> None:
raise_giant_error()
this results in the task being marked as FAILED showing the full giant error in the UI, but the workflow remains stuck in RUNNING and the FlytePropeller logs show Failed to update workflow. Error [rpc error: code = ResourceExhausted desc = trying to send message larger than max (2805993 vs. 2097152)]
. This is not solved by adjusting the maxMessageSizeBytes
or UseOffloadedWorkflowClosure
.
please let me know if there is more information I can provide!freezing-airport-6809
hallowed-mouse-14616
11/06/2023, 3:05 PMabundant-laptop-47033
11/06/2023, 4:34 PMabundant-laptop-47033
11/06/2023, 4:42 PM