https://flyte.org logo
#ask-the-community
Title
# ask-the-community
r

Rafael Raposo

03/27/2024, 2:59 PM
Hey friends. Quick question, I've got a workflow using
map_task
and the launchplan is using
raw_output_data_config
but we still have map task failing with
Copy code
caused by: failed to execute handle for plugin [k8s-array]: output file @[gs://..../outputs.pb]
      is too large [13679556] bytes, max allowed [10485760] bytes'
Is the output offloading thing not supported in
map_task
?
h

Haytham Abuelfutuh

03/27/2024, 4:24 PM
The issue is, likely, caused by the aggregate output size of all subtasks of a
map_task
... Even if each subtask is offloaded, the metadata output of that will contain the URL of the output of the subtask, when you put 10K URLs together, they add up... (13mb it seems)
r

Rafael Raposo

03/27/2024, 4:38 PM
ahah thanks @Haytham Abuelfutuh
2 Views