https://flyte.org logo
Powered by
s

Sharon Gong

04/14/2022, 7:19 PM
hi there! hoping someone can help me out with launching a workflow. I'm able to launch a workflow in flytectl that I'm unable to using flytekit's 
FlyteRemote
. for some context, this workflow was created by copying over the 
PythonFunctionWorkflow
and it redefines the workflow's module and name. details in 🧵
the error I get from 
FlyteRemote
is this
Copy code
KeyError(resource_type: LAUNCH_PLAN
project: "sp-one-model"
domain: "staging"
name: "onemodel.models.subscribers.onemodel"
version: "2eb9f303-a29e-42e7-909f-21ea0916e56c"
)
where the stacktrace is
Copy code
File "/Users/sharong/repo/jasper-api/backend/jasper_api/routes.py", line 235, in execute
    lp = remote.fetch_launch_plan(project, domain, workflow, version)
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/remote.py", line 443, in fetch_launch_plan
    workflow = self.fetch_workflow(wf_id.project, wf_id.domain, wf_id.name, wf_id.version)
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/remote.py", line 414, in fetch_workflow
    return FlyteWorkflow.promote_from_closure(compiled_wf, node_launch_plans)
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/workflow.py", line 159, in promote_from_closure
    tasks=tasks,
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/workflow.py", line 106, in promote_from_model
    for node in base_model_non_system_nodes
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/workflow.py", line 106, in <dictcomp>
    for node in base_model_non_system_nodes
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/nodes.py", line 89, in promote_from_model
    tasks,
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/component_nodes.py", line 119, in promote_from_model
    tasks=tasks,
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/workflow.py", line 106, in promote_from_model
    for node in base_model_non_system_nodes
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/workflow.py", line 106, in <dictcomp>
    for node in base_model_non_system_nodes
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/nodes.py", line 89, in promote_from_model
    tasks,
  File "/Users/sharong/.pyenv/versions/3.7.4/envs/jasper-api/lib/python3.7/site-packages/flytekit/remote/component_nodes.py", line 108, in promote_from_model
    base_model.launchplan_ref, node_launch_plans[base_model.launchplan_ref]
I compared the workflow against the original workflow and I noticed the attributes 
lhs
and 
location
were not defined. it seems like 
lhs
needs to be defined - can you elaborate on what the attribute is for and how it's defined?
cc @Dylan Wilder
y

Yee

04/14/2022, 7:55 PM
hey can you share the code please?
feel free to redact as much as you need… just want to see the structure of it
also which flytekit version are you on?
s

Sharon Gong

04/14/2022, 7:57 PM
of how we're using flytekit? it's just this and it breaks during the fetch. we're using 
flytekit==0.23.0
and i've also tried 
flytekit==0.25.0
Copy code
remote = FlyteRemote.from_config(config_file_path='flyte.config')
        lp = remote.fetch_launch_plan(project, domain, workflow, version)
d

Dylan Wilder

04/14/2022, 9:30 PM
hey i think we've tracked this down @Yee to an issue unrelated to our code. imaging this DAG: wf_a -> wf_b -> lp_c where 
->
translates to invokes as in wf_b is a subworkflow of a.
from flytectl we can fetch/launch lp_c, wf_b, and wf_a all just fine
from flyteremote we can fetch lp_c and wf_b but not wf_a which fails with a key error on lp_c
if however, we use wf_c instead of lp_c there are no issues
last observation, the chain wf_a -> lp_b -> lp_c works fine 😄
in plain english: workflows can invoke a launch plan fine, but if there is a third level of recursion (workflow invokes workflow invokes launch plan) we run into issues
y

Yee

04/14/2022, 9:36 PM
sorry was afk…
d

Dylan Wilder

04/14/2022, 9:38 PM
no worries i just blasted you with data
y

Yee

04/14/2022, 9:38 PM
just kinda confused.
d

Dylan Wilder

04/14/2022, 9:39 PM
yea i tried my best over slack, can jump on a quick call maybe
y

Yee

04/14/2022, 9:39 PM
when you say wf_a, wf_b and lp_c… are these locally defined python objects? or fetched entities? or does it not matter
yeah i’m down
let me hop into a room real quick
i’m gonna add some more tests
but that is the root of the issue
and from the todo clearly branch nodes don’t quite work yet either, but will submit that as a separate PR.
d

Dylan Wilder

04/19/2022, 2:58 PM
what version will this be released under?
y

Yee

04/19/2022, 3:57 PM
not sure yet. probably 0.32.6
4 Views
Powered by