https://flyte.org logo
#ask-the-community
Title
# ask-the-community
d

Dennis Keck

07/27/2023, 4:39 PM
Hi all, we saw a weird issue with loading a task cache if there is a nested list, e.g.:
Copy code
from flytekit import workflow, task

@task(
    cache=True,
    cache_version="0"
)
def my_task() -> list[list[str]]:
    return [["foo"], []]

@workflow
def my_workflow():
    my_task()
Executing this a second time, the task will be stuck in
Queued
and there is no error message shown to the user. The flytepropeller logs indicate that it interprets the type of the cached object wrongly:
Copy code
Error when trying to reconcile workflow. Error [Failed to check Catalog for previous results: unexpected artifactData: [o0] type: [collection_type:<union_type:<variants:<collection_type:<simple:NONE > > variants:<collection_type:<simple:STRING > > > > ]
I assume this is is bug in flytepropeller and created an issue.
d

Dan Rammer (hamersaw)

07/27/2023, 4:53 PM
quick question, was this task executed with a different return type originally and then updated the return type without bumping the
cache_version
? i know the input interface is included in the cache key, but i'm not sure if the outputs are. this error seems to say "we're expected type N because that is what is stored in the cache, but the task output is actually type M"
but i now exactly where this is occuring and can hopefully take a look in the next few days.
d

Dennis Keck

07/27/2023, 4:59 PM
When I debugged it with our complex nested & dynamic workflow I expected something like this first too. Now I was able to reproduce it though with this very minimal example where I started from a clean plate. To be 100% sure I also raised the cache version once more and tried it again. Same behavior: writing the cache works, reading it fails.
d

Dan Rammer (hamersaw)

07/27/2023, 5:00 PM
oh sure, certainly a bug then! thanks for filing with such depth, i'll try to look asap.
d

Dennis Keck

07/27/2023, 5:02 PM
Thanks, Dan! We are trying to work around it by nesting our list. So probably no urgency. It just took me a while to figure out the root cause 😅
4 Views