https://flyte.org
Join Slack
Any suggestions on how to invalidate the cache when running a map task? Seeing <this bug> where task...
# flyte-support
c
Any suggestions on how to invalidate the cache when running a map task? Seeing this bug where tasks complete immediately even though I've tried: 1. disabling caching by setting 
cache=False
on the mapped task 2. bumping the cache version from "0.1" to "0.2" on the mapped task
m
Can you register a new version and run ?
f
Use experimental.map_task
c
I tried a new version, that didn't work. Ended up breaking one of the tasks in my workflow into two (needed to do that anyway) so that seemed to invalidate the cache.
Ketan, I was having an issue with map_task that could only be resolved by wrapping it in a dynamic task.
Copy code
@task
def t(a: int) -> int:
    return a + 1
    
@dynamic
def t_wrapper(xs: List[int]) -> None:
    map_task(t)(a=xs)

@workflow
def experimental_map_task(xs: List[int]) -> None:
    t_wrapper(xs=xs)
t
What's the error you're seeing, @calm-zoo-68637?
c
If I don't do that strange wrapper thing, then when I try to register the task I get:
Copy code
Error 0: Code: ParameterNotBound, Node Id: end-node, Description: Parameter not bound [o0]. Error 1: Code: ValueRequired, Node Id: n2, Description: Value required [Target]. Error 2: Code: ValueRequired, Node Id: n3, Description: Value required [Target]. Error 3: Code: VariableNameNotFound, Node Id: n2, Description: Variable [o0] not found on node [n2]
t
Are all your flyte components updated to 1.9.1?
c
I'm using flytekit 1.9.0
t
1.9.0 should work as well. @hallowed-mouse-14616, what could be the cause of this error?
h
will get to looking into this - seems to potentially be a flytekit issue where the dynamic is executed with a different version?!
f
This is the same question Tom is also asking
h
and others
f
I can help from flytekit side
👍 1
h
@calm-zoo-68637 how are you registering this? flytectl?
f
@hallowed-mouse-14616 is this the same problem as hashing? If not can you share an example I can help
c
ended up just passing a nonce to the task as an input to overwrite cache. Yes registering with 
flytectl register
f
Ohh is this the same as the older version of admin
h
> Ohh is this the same as the older version of admin Yes, @calm-zoo-68637 for some context - https://flyte-org.slack.com/archives/CP2HDHKE1/p1697574227413249?thread_ts=1696568069.202449&cid=CP2HDHKE1. TL;DR the 1.10 release will support ArrayNode.
🙌 1
7 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by