https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
g

Gopal Vashishtha

10/12/2023, 1:15 AM
Any suggestions on how to invalidate the cache when running a map task? Seeing this bug where tasks complete immediately even though I've tried: 1. disabling caching by setting 
cache=False
on the mapped task 2. bumping the cache version from "0.1" to "0.2" on the mapped task
j

Jay Ganbat

10/12/2023, 2:43 AM
Can you register a new version and run ?
k

Ketan (kumare3)

10/12/2023, 3:29 AM
Use experimental.map_task
g

Gopal Vashishtha

10/12/2023, 4:53 AM
I tried a new version, that didn't work. Ended up breaking one of the tasks in my workflow into two (needed to do that anyway) so that seemed to invalidate the cache.
Ketan, I was having an issue with map_task that could only be resolved by wrapping it in a dynamic task.
Copy code
@task
def t(a: int) -> int:
    return a + 1
    
@dynamic
def t_wrapper(xs: List[int]) -> None:
    map_task(t)(a=xs)

@workflow
def experimental_map_task(xs: List[int]) -> None:
    t_wrapper(xs=xs)
s

Samhita Alla

10/12/2023, 6:48 AM
What's the error you're seeing, @Gopal Vashishtha?
g

Gopal Vashishtha

10/12/2023, 8:04 AM
If I don't do that strange wrapper thing, then when I try to register the task I get:
Copy code
Error 0: Code: ParameterNotBound, Node Id: end-node, Description: Parameter not bound [o0]. Error 1: Code: ValueRequired, Node Id: n2, Description: Value required [Target]. Error 2: Code: ValueRequired, Node Id: n3, Description: Value required [Target]. Error 3: Code: VariableNameNotFound, Node Id: n2, Description: Variable [o0] not found on node [n2]
s

Samhita Alla

10/12/2023, 8:58 AM
Are all your flyte components updated to 1.9.1?
g

Gopal Vashishtha

10/12/2023, 2:52 PM
I'm using flytekit 1.9.0
s

Samhita Alla

10/12/2023, 2:57 PM
1.9.0 should work as well. @Dan Rammer (hamersaw), what could be the cause of this error?
d

Dan Rammer (hamersaw)

10/16/2023, 2:57 PM
will get to looking into this - seems to potentially be a flytekit issue where the dynamic is executed with a different version?!
k

Ketan (kumare3)

10/16/2023, 3:06 PM
This is the same question Tom is also asking
d

Dan Rammer (hamersaw)

10/16/2023, 3:07 PM
and others
k

Ketan (kumare3)

10/16/2023, 3:07 PM
I can help from flytekit side
d

Dan Rammer (hamersaw)

10/16/2023, 4:16 PM
@Gopal Vashishtha how are you registering this? flytectl?
k

Ketan (kumare3)

10/17/2023, 2:15 PM
@Dan Rammer (hamersaw) is this the same problem as hashing? If not can you share an example I can help
g

Gopal Vashishtha

10/19/2023, 3:17 AM
ended up just passing a nonce to the task as an input to overwrite cache. Yes registering with 
flytectl register
k

Ketan (kumare3)

10/19/2023, 3:22 AM
Ohh is this the same as the older version of admin
d

Dan Rammer (hamersaw)

10/19/2023, 1:12 PM
> Ohh is this the same as the older version of admin Yes, @Gopal Vashishtha for some context - https://flyte-org.slack.com/archives/CP2HDHKE1/p1697574227413249?thread_ts=1696568069.202449&cid=CP2HDHKE1. TL;DR the 1.10 release will support ArrayNode.
2 Views
Powered by