<#381 [BUG] Flyte List objects should be offloaded...
# flyte-github
c
#381 [BUG] Flyte List objects should be offloaded or offloadable. Also In case of large outputs, propeller should return an error Issue created by kumare3 Describe the bug For an example workflow
Copy code
{"json":{"exec_id":"z4b5bbelyu","node":"write-task","ns":"...-development","res_ver":"1085530040","routine":"worker-357","src":"plugin_manager.go:257","tasktype":"spark","wf":"...."},"level":"info","msg":"Plugin [spark] returned no outputReader, assuming file based outputs","ts":"2020-07-08T21:11:03Z"}
{"json":{"exec_id":"z4b5bbelyu","node":"write-task","ns":"...-development","res_ver":"1085530040","routine":"worker-357","src":"pre_post_execution.go:113","tasktype":"spark","wf":".."},"level":"error","msg":"Failed to check if the output file exists. Error: error file @[s3://..../error.pb] is too large [13117404] bytes, max allowed [10485760] bytes","ts":"2020-07-08T21:11:03Z"}
This is actually the output is too large. But the workflow gets stuck instead of failing instantly unable to load the file Expected behavior Failure should be propagated to admin with the size restrictions Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☑︎ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other To Reproduce Steps to reproduce the behavior: 1. ... 2. ... Screenshots ... Additional context The workflows continue to run to eventually result in system error, while this is actually a user error. flyteorg/flyte