Hey team we're running map tasks and seeing a lot ...
# flyte-supportd
delightful-queen-21464
03/18/2025, 11:50 PMHey team we're running map tasks and seeing a lot of propeller pod crashlooping due to panics from some variation of
concurrent map writesv1.13.3delightful-queen-21464
03/18/2025, 11:51 PMUntitled
f
freezing-airport-6809
03/19/2025, 1:24 AM
I think this was fixed in a later version?
h
high-park-82026
03/19/2025, 1:28 AM
Checking
d
delightful-queen-21464
03/19/2025, 2:45 PMgreat, any version we should update to? I found this https://github.com/flyteorg/flyte/issues/6038 but it was related to the flyteadmin, not propeller
delightful-queen-21464
03/19/2025, 10:03 PMany update here? @high-park-82026
delightful-queen-21464
03/24/2025, 4:51 PMjust to add on here, we are seeing issues that may be related to the panics, as they are definitely happening on the map tasks as well. these workflows fail with
1.
RuntimeExecutionError: max number of system retry attempts [51/50] exhausted. Last known status message: AlreadyExists: Event already exists, caused by [rpc error: code = AlreadyExists desc = have already recorded task execution phase RUNNING (version: 44) for task_idRuntimeExecutionError: max number of system retry attempts [61/60] exhausted. Last known status message: [system] unable to read futures file, maybe corrupted, caused by: [system] Failed to read futures protobuf filedelightful-queen-21464
03/24/2025, 4:52 PMcc: @many-salesclerk-79629
2 Views