https://flyte.org logo
#ask-the-community
Title
# ask-the-community
z

Ziyad Sheebaelhamd

11/17/2023, 5:07 PM
Hi all, I am executing a flyte map_task, and the caching does not kick in when re-executing the workflow. My task looks as follows:
Copy code
@task(cache=True, cache_version="1.0", requests=Resources(mem="12Gi", cpu="1"), retries=1)
def write_data(cfg: DictConfig, key: str) -> None:
   ....
and I am parallelizing over keys in my workflow as follows:
Copy code
partial_task = partial(write_data, cfg=cfg)
map_task(partial_task, concurrency=30, min_success_ratio=0.8)(key=keys)
I am using flytekit version
1.10.0
j

Jay Ganbat

11/17/2023, 6:25 PM
you might have to explicitly set the metadata in the map_task
Copy code
metadata=TaskMetadata(
        cache=True,
        cache_version=VERSION)
z

Ziyad Sheebaelhamd

11/21/2023, 11:46 AM
Hi @Jay Ganbat, thanks for the proposed solution. I attempted explicitly setting the metadata, but it did not work. I am getting
caching was disabled for this execution
in the flyte console:
k

Kevin Su

11/22/2023, 12:07 AM
@Ziyad Sheebaelhamd mind creating an issue here? [flyte-bug]
k

Kevin Su

11/22/2023, 12:07 AM
btw, which version of flyte you are using?
j

Jay Ganbat

11/22/2023, 12:26 AM
pretty old version, posted above, 1.10
d

Dan Rammer (hamersaw)

11/27/2023, 10:50 AM
@Ziyad Sheebaelhamd do you see cache indicators for individual subtasks? In maptasks the top-level item as you've depicted in the UI will never show a cached indicator, rather each individual subtask has it's own indicators.