Any suggestions on how to invalidate the cache whe...
# flyte-supportc
calm-zoo-68637
10/12/2023, 1:15 AMAny suggestions on how to invalidate the cache when running a map task? Seeing this bug where tasks complete immediately even though I've tried:
1. disabling caching by setting
cache=Falsem
magnificent-teacher-86590
10/12/2023, 2:43 AMCan you register a new version and run ?
f
freezing-airport-6809
10/12/2023, 3:29 AM
Use experimental.map_task
c
calm-zoo-68637
10/12/2023, 4:53 AMI tried a new version, that didn't work. Ended up breaking one of the tasks in my workflow into two (needed to do that anyway) so that seemed to invalidate the cache.
calm-zoo-68637
10/12/2023, 4:54 AMKetan, I was having an issue with map_task that could only be resolved by wrapping it in a dynamic task.
Copy code
@task
def t(a: int) -> int:
return a + 1
@dynamic
def t_wrapper(xs: List[int]) -> None:
map_task(t)(a=xs)
@workflow
def experimental_map_task(xs: List[int]) -> None:
t_wrapper(xs=xs)t
tall-lock-23197
10/12/2023, 6:48 AM
What's the error you're seeing, @calm-zoo-68637?
c
calm-zoo-68637
10/12/2023, 8:04 AMIf I don't do that strange wrapper thing, then when I try to register the task I get:
Copy code
Error 0: Code: ParameterNotBound, Node Id: end-node, Description: Parameter not bound [o0]. Error 1: Code: ValueRequired, Node Id: n2, Description: Value required [Target]. Error 2: Code: ValueRequired, Node Id: n3, Description: Value required [Target]. Error 3: Code: VariableNameNotFound, Node Id: n2, Description: Variable [o0] not found on node [n2]t
tall-lock-23197
10/12/2023, 8:58 AM
Are all your flyte components updated to 1.9.1?
c
calm-zoo-68637
10/12/2023, 2:52 PMI'm using flytekit 1.9.0
t
tall-lock-23197
10/12/2023, 2:57 PM
1.9.0 should work as well. @hallowed-mouse-14616, what could be the cause of this error?
h
hallowed-mouse-14616
10/16/2023, 2:57 PMwill get to looking into this - seems to potentially be a flytekit issue where the dynamic is executed with a different version?!
f
freezing-airport-6809
10/16/2023, 3:06 PM
This is the same question Tom is also asking
h
hallowed-mouse-14616
10/16/2023, 3:07 PMand others
f
freezing-airport-6809
10/16/2023, 3:07 PM
I can help from flytekit side
👍 1
h
hallowed-mouse-14616
10/16/2023, 4:16 PM@calm-zoo-68637 how are you registering this? flytectl?
f
freezing-airport-6809
10/17/2023, 2:15 PM
@hallowed-mouse-14616 is this the same problem as hashing? If not can you share an example I can help
c
calm-zoo-68637
10/19/2023, 3:17 AMended up just passing a nonce to the task as an input to overwrite cache. Yes registering with
flytectl registerf
freezing-airport-6809
10/19/2023, 3:22 AM
Ohh is this the same as the older version of admin
h
hallowed-mouse-14616
10/19/2023, 1:12 PM> Ohh is this the same as the older version of admin
Yes, @calm-zoo-68637 for some context - https://flyte-org.slack.com/archives/CP2HDHKE1/p1697574227413249?thread_ts=1696568069.202449&cid=CP2HDHKE1. TL;DR the 1.10 release will support ArrayNode.
🙌 1
5 Views