Hi! I have LaunchPlan created in Python that I want to trigger in my CI/CD flow based on file changes. The documentation shows how to create an execution given a spec file, but how do I actually execute a launch plan that’s already created programmatically?

You can use flytekit.remote

Ok thanks. It’s not possible to use any of the CLI tools?

Hmm you can use flytectl

Can you give an example? I know how to get launch plans and create executions, but execution files are clunky for CI/CD. I basically just want to execute a launch plan with some dynamic inputs. I could just execute the workflow directly but launch plans have some other nice benefits, like notifications.

flytectl is just pulling the default launch plan btw for the workflow.

I think so

Maybe I’m not using the Flyte API the correct way. My use case is the following: I am monitoring a folder of metadata files for different processes. Whenever these change, I want to trigger a Workflow with the changed process as input. This all works in my GitHub action, but it would be nice to execute a pre-existing LaunchPlan instead of a Workflow. Does this make sense?

So the question here is that we have one model architecture that we want to train and fine tune for many different datasets, lets say we have N of them. Ideally, we would like to have one workflow that produces a model, use N different launch plans, one for each dataset, to trigger a specific training for each model. The launch plan it self could be the same for each dataset, except for one, or many, parameters that should vary based on dataset. Similar to what you do with overriding values in Helm installations for instance. How do we structure this? We just want to define the workflow once in code, and then use variables to train it for a specific dataset. Let me know if it is not clear and I can explain further 🙂

@faint-monitor-96441 / @helpful-crowd-74546 mostly makes sense, But some confusion- let me paraphrase- You want to trigger a workflows with different dataset? If so why can you not just use different input values. Launchplans are useful if you want to change the interface of a workflow- default some inputs and or schedule them- as scheduling can only provide time as an input

If so why can you not just use different input values.

How would you then do that? I think the confusion we have is that we don’t really know how to achieve different input values for a workflow in an automated way e.g., in CI/CD workflow. In the UI it is trivial

Ohh you just don’t know how to send different inputs?

No,

pyflyte

can’t run with

LaunchPlan

, at least it’s not documented

just following up on this, I opened https://github.com/flyteorg/flyte/issues/2998 to track the work to enable launchplans in pyflyte run. Should be a pretty simple contribution. 🙂