https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
l

Laura Lin

10/06/2023, 4:54 AM
Copy code
Error 0: Code: ParameterNotBound, Node Id: end-node, Description: Parameter not bound [o0].
	Error 1: Code: ValueRequired, Node Id: n0, Description: Value required [Target].
	Error 2: Code: VariableNameNotFound, Node Id: n0, Description: Variable [o0] not found on node [n0].
When I try to use the experimental map_nodes on 1.9.1, I get the following errors. this is with
Copy code
from typing import List
from flytekit import task, workflow
from flytekit.experimental import map_task

@task
def t(a: int) -> int:
    return a + 1
    
@workflow
def experimental_map_task(xs: List[int]) -> List[int]:
    return map_task(t)(a=xs)
looks like there was another thread here https://flyte-org.slack.com/archives/CP2HDHKE1/p1695066063355849?thread_ts=1694733863.990439&cid=CP2HDHKE1 that saw a similar error trying to switch over
I'm using flyte-core not flyte-binary
s

Samhita Alla

10/06/2023, 5:40 AM
cc @Dan Rammer (hamersaw)
l

Laura Lin

10/06/2023, 4:08 PM
yea not sure; I dug around looking at flyteadmin logs but don't really see anything. Was there a helm values change needed or some flag to enable?
d

Dan Rammer (hamersaw)

10/06/2023, 5:54 PM
@Laura Lin it's not clear to me that the other thread is related. I know we saw this error with another user who only updated flytekit to 1.9.1, but you also need to update flytepropeller and flyteadmin for ArrayNode support.
l

Laura Lin

10/06/2023, 5:55 PM
my helm chart is on 1.9.1 and I have flytekit==1.9.1 locally
I can see that the docker images they are using are from 1.9.1 release
bump on this. seeing this in a new deployment that is on 1.9.1 release too
strange...this works
Copy code
@task
def t(a: int) -> int:
    return a + 1
    
@dynamic
def t_wrapper(xs: List[int]) -> None:
    map_task(t)(a=xs)

@workflow
def experimental_map_task(xs: List[int]) -> None:
    t_wrapper(xs=xs)
d

Dan Rammer (hamersaw)

10/16/2023, 2:58 PM
@Laura Lin it looks like this issue is not isolated. Thanks for your patience on resolving (took some time off to reload), but will make this a priority.
Are you registering this using flytectl? If so, do you know what version?
I have tried deploying 1.9.1 using helm charts and flytectl demo cluster locally and am unable to reproduce this. The error is coming from the flytepropeller compiler. This code runs in two separate locations, first within FlyteAdmin during workflow registration and second, in FlytePropeller for dynamic workflow compilation. The fact that this fails during workflow registration leads me to believe that the flyteadmin version does not have the latest compiler updates to support ArrayNode. However, this doesn't make sense given that admin is on v1.9.1.
Have you tried this on a local demo cluster with 1.9.1?
Copy code
./bin/flytectl demo start
and then use your flytekit to execute the ArrayNode. Are there any issues here?
@Gopal Vashishtha interested if you are getting this same error using the 1.9.1 flytectl demo cluster as well? It might help debug this to get it all in one thread.
l

Laura Lin

10/16/2023, 7:14 PM
@Dan Rammer (hamersaw) do you want to hop into a call to take a look at the setup, might be easier for you to look at all the parts you want to see
oh... I can't reproduce it using the demo cluster.
oh i can't reproduce it anymore in my current cluster either. 🤔 maybe i toggled something randomly was using the latest flyteadmin image and not the 1.9.1 release
d

Dan Rammer (hamersaw)

10/16/2023, 7:33 PM
not sure if this is a good or bad thing 😅. was this happening using 
pyflyte run ...
, or 
pyflyte register ...
, or 
flytectl register ...
?
l

Laura Lin

10/16/2023, 7:33 PM
its still breaking on another deployment tho.
Copy code
flytectl register
k

Ketan (kumare3)

10/17/2023, 2:48 PM
Hmm flytectl register may not handle the new node type @Dan Rammer (hamersaw) ?
d

Dan Rammer (hamersaw)

10/17/2023, 3:22 PM
Hmm flytectl register may not handle the new node type
It does and the error would be different - something about hydrating a nil nodespec.
k

Ketan (kumare3)

10/17/2023, 4:08 PM
admin does not have the new compiler
d

Dan Rammer (hamersaw)

10/17/2023, 8:23 PM
So update here for visibility. The issue here is that the Flyte compiler requires an update to support ArrayNode. This code resides in the flytepropeller repo and is used in flyteadmin during workflow registration and flytepropeller when processing dynamic nodes. The single binary and individual component releases were being cut from different repos (namely the github.com/flyteorg/flyte and github.com/flyteorg/flyteadmin repos respectively for this scenario). Unfortunately there was an oversight in updating the flytepropeller verison in the github.com/flyteorg/flyteadmin repo to support ArrayNode by updating the compiler. This means that the 1.9.0 and 1.9.1 release for single binary can support ArrayNode, but the individual component releases will not. The move to a monorepo will ensure that all dependencies are identical regardless of releasing as a single binary or individual component, so this will not be an issue moving forward.
l

Laura Lin

10/17/2023, 8:25 PM
thanks for the update @Dan Rammer (hamersaw), appreciate the help!
Are we planning on patching the 1.9.1 release? or run 1.9.2 non-release tag of some sort
Or I should just wait for the 1.10.0.
k

Ketan (kumare3)

10/18/2023, 11:08 PM
We are releasing 1.10 almost there - maybe tomorrow
Powered by