Hi there! Anyone have ideas on what can cause a

task

to succeed ✅ but never run/schedule a pod ❌? Or know where to start troubleshooting? 🕵️‍♂️ -- More details -- The task in question has caching disabled, and is a

map

task. Have searched all flyte component logs for the executions and the only errors I see are these warning ⚠️ messages

Failed to fetch override values when assigning task resource default values

Failed to fetch override values when assigning execution queue

Trying to disable the overrides didn't help either. And the task repeatedly succeeds without ever running the task. cc: @Heidi Hurst

Without caching enabled this is not possible- are you are the subtasks don’t have cache enabled

Hm good to know, we see

Caching was disabled for this execution.

in the console for the task in question That being said, we did not disable in the task decorator, but via the new

Overwrite cached outputs

flag. Perhaps that may not be working properly...

Ah, ok. So it sounds like the task definition has caching enabled, You are using it in a map task with something like:

@task(cache=True, cache_version="1.0")
def a_mappable_task

@workflow
def my_map_workflow(a: List[int]):
    mapped_out = map_task(a_mappable_task)(a=a)

And then when you execute from the UI (or elsewhere) you're selecting to overwrite the cache values?

yep exactly!

If so, this is a construct of how map tasks are implemented. Namely, as a separate plugin rather than a core Flyte component. The "cache overwrite" functionality is not currently not support in map tasks. It is possible to integrate it - would you mind filing a bug describing how cache overwrite does not work with map tasks? [flyte-bug]

:ladybug: Create a new Flyte Bug issue: https://github.com/flyteorg/flyte/issues/new?assignees=&labels=bug%2Cuntriaged&template=bug_report.yaml&title=%5BBUG%5D+

but your current execution should be just reading the values from the cache right?

@Dan Rammer (hamersaw) we need to find you time

Ah I see..good to know that it's not supported. Will file an issue! If that's the case then I guess yeah it may still be reading from cache. We will try disabling cache in the task decorator and test. Thanks you two 🙏 Issue filed