https://flyte.org logo
#ask-the-community
Title
# ask-the-community
g

Gauthier Castro

11/02/2023, 8:39 AM
Hi again, similar question about naming. How can I rename a workflow? We have a use case where we’ll need to maintain 100s of different instances of the same workflow (training ML models out of the same training pipeline) and are looking into what would be the best way to achieve that in flyte.
s

Samhita Alla

11/03/2023, 7:06 AM
I'm afraid that's not possible. But I'd like to understand more — do you want to trigger the same workflow multiple times?
g

Gauthier Castro

11/06/2023, 10:45 AM
I think we’ll go with a solution where we import tasks from another pipeline (or pipeline) and register as a new “clone” workflow then. We have a project where we have to train 100s of small (and usually simple) models with the same training pipeline. We curate and refine the dataset locally then plug it into the flyte training workflow. The workflow generates a packaged model as well as performance metrics. We typically train 6 of these models per week and revisit them when the live perf start degrading. That means that having them all under the same workflow but different runs wouldn’t be practical, especially on the UI. We think that creating a new workflow for each would help browsing through different models. Browsing through each experiment on a given models would rather easy as well with each run of the workflow. If renaming workflows isn’t possible we’re likely to setup our repo so that we have something like
dags/<group name>/<model name>/training.py
and it should give us workflows with name
dags.<group name>.<model name>.<http://training.wf|training.wf>
s

Samhita Alla

11/06/2023, 1:24 PM
Makes sense. You can given an execution a name, however. Here's an example:
pyflyte run --remote --name <execution-name> ...
g

Gauthier Castro

11/06/2023, 1:26 PM
Thanks, I thoughout about this but wasn’t sure how we could version it. The only way I could think of was in the name such as
my_group-my_model-v1.0.1
as the version would be for the workflow not the execution, right?
s

Samhita Alla

11/06/2023, 1:27 PM
Yes, version is for a workflow.
You don't need to append a version to your execution name as you should be able to filter executions corresponding to a specific version on the UI with ease.
g

Gauthier Castro

11/06/2023, 1:31 PM
That’s where I guess our use case is a bit deeper. We share a workflow and it can have versions. Then each model (
my_group-my_model
) will have different experiments (run/executions) with different input parameters (to the workflow)
s

Samhita Alla

11/06/2023, 1:32 PM
Understood. So what's the end goal? Does an execution name not suffice?
g

Gauthier Castro

11/06/2023, 1:35 PM
Ultimately we want to be able to browse based on • Workflow version • Group name • Model name • Model experiment (run with different inputs) We from what I’ve seen on flyte would align better with having 1 workflow per group_name/model_name and each run under a workflow would be an experiment for that model. Can we get that directly with execution name? I can’t see how to filter/browse the executions other than just the list from the UI.
here’s an example on what I ran on friday
s

Samhita Alla

11/06/2023, 1:56 PM
You cannot browse by an execution name; you'll have to go through the list of executions. Mind creating an issue?
g

Gauthier Castro

11/06/2023, 1:59 PM
Yeah sure, I’ll do that later today. List of executions is fine with each model has its workflow and having execution as an experiment for a given model. But if we group all the models under the same workflow it becomes a bit difficult indeed
s

Samhita Alla

11/06/2023, 2:01 PM
Yeah, grouping models under the same workflow leads to confusion. You can add tags to your executions, but I don't think this feature is available on the UI yet. @Kevin Su, any idea when this is going to be implemented or if it's available already?
g

Gauthier Castro

11/07/2023, 2:03 PM
After reading again bits of the documentation (especially this section) I feel that maybe launch plans are the way to go after all…
s

Samhita Alla

11/07/2023, 2:13 PM
Yeah! Launch plans can be used as well. You can create multiple launch plans for the same workflow.
g

Gauthier Castro

11/07/2023, 2:31 PM
s

Samhita Alla

11/07/2023, 2:35 PM
Wouldn't launch plans solve your problem? You can have a single workflow and multiple launch plans. That essentially means renaming the workflow as per your experiment, correct?
g

Gauthier Castro

11/07/2023, 2:38 PM
I feel like launch plans may not make it easy to track experiments. If we use a launch plan for each model, we’d create a new version of the launch plan for each experiment I think? And no, renaming workflow wouldn’t be per experiment. Instead of launch plan, each model would have its own workflow and experiments would be execution of the workflow (with the automatic random name when no name is given)
If I wanted to browse experiments and scenario 1 is with launch plans and scenario 2 is renamed workflows 1. I’d go on the launch plan UI, search for a model/launch plan by name with the search bar.and browse experiments by browsing different versions of the plan 2. I’d go on the workflow UI, search. for model/workflow plan by name and browse experiments by browsing through the execution list Is my understanding correct? Perhaps we could also use executions in the launch plan but not quite sure how it differs from the versions.
s

Samhita Alla

11/07/2023, 2:44 PM
You'll have executions under a launch plan. It has the same interface as that of a workflow, but launch plan has executions which have been triggered via that specific launch plan.
It's executions in both workflow and launch plan.
g

Gauthier Castro

11/07/2023, 2:53 PM
Could be the way to go then! And trigger the launch plan programmatically with the right inputs in a small python script. What happens when we need to change the workflow? I already think of the 2 use cases here: 1. same workflow, new version of the workflow (Say we add a little feature to the workflow or fix a bug) 2. other workflow (context of the models changes and we have to use an alternative training pipeline)
s

Samhita Alla

11/08/2023, 6:59 AM
And trigger the launch plan programmatically with the right inputs in a small python script.
You can trigger launch plans programmatically using FlyteRemote. Say you update your workflow, and there's a launch plan associated with it already, you need to reregister your launch plan with a new version.
@Gauthier Castro here's the launch plan and workflow I registered: https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/basics/launch_plan.html. I ran the command
pyflyte register launch_plan.py
and that registered the launch plans, tasks and workflows in the
workflow.py
file (the workflow is being imported from the
workflow.py
file) The workflow was modified in the
workflow.py
file, and I re-registered the entities using the same command. Upon launching the workflow in the UI using the pre-defined launch plan, the updated workflow was triggered as expected.
Is this something you tried, and does this look relevant to your use-case?
g

Gauthier Castro

11/08/2023, 1:17 PM
That definitely looks relevant! I must have done something slightly different because it kept on triggering the “old” workflow. I’ll give that a go
It kind of works but it also registers a new version of everything which isn’t quite what I’d like. When I try with the python API it doesn’t re-register everythng though. So I might be able to get the behaviour I’m after! However there may be a little “inconsistency” on the UI. 1. Register a new version of existing launch plan with a new workflow (completely different one, not just a different version of the same workflow) 2. Go to workflows, find the new workflow 3. Click launch and select the launch plan version freshly registered 4. Go to launch plans and select the launch plan 5. Click the execution you just triggered Then at the top of the UI the workflow name will still be the old workflow. It definitely executing the new workflow though. It’s probably just a UI thing because neither the execution nor the version are listed in the old workflow but they are in the new workflow.
Just to confirm, there isn’t any way to trigger a launch plan from the launch plan UI, right? You have to go to the workflow section and launch with the launch plan you wish to use, right?
s

Samhita Alla

11/09/2023, 7:22 AM
> Then at the top of the UI the workflow name will still be the old workflow. Is it possible for you to share some screenshots? This may be a bug, but want to confirm first. > Just to confirm, there isn’t any way to trigger a launch plan from the launch plan UI, right? You have to go to the workflow section and launch with the launch plan you wish to use, right? Yes, launch plan is just a plan to launch a workflow. And hence, it's indeed launching a workflow, but not a launch plan.
g

Gauthier Castro

11/09/2023, 10:18 AM
1. From the launch plan, click execution
abcv46ptw8mxwgr5zv9r
2. In the execution page, click on the workflow name at the top (which should have been dag.py after the change)
3. In the workflow page you can see that the execution id
abcv46ptw8mxwgr5zv9r
is not there neither the version 0.2.15
4. However if I go to workflow
dag.py
I can see the version 0.2.15 and the execution id
abcv46ptw8mxwgr5zv9r
Interestingly enough if from 4. on the dag.py workflow I click on the execution id
abcv46ptw8mxwgr5zv9r
it takes me back to screenshot 2 with the wrong workflow name!
> Just to confirm, there isn’t any way to trigger a launch plan from the launch plan UI, right? You have to go to the workflow section and launch with the launch plan you wish to use, right?
Yes, launch plan is just a plan to launch a workflow. And hence, it’s indeed launching a workflow, but not a launch plan. (edited)
may be a good feature request to allow executing a launch plan with its config from the launch plan UI. It’s pointing to a workflow after all 🙂 so it may be possible to work out which workflow to execute
s

Samhita Alla

11/09/2023, 1:22 PM
Interestingly enough if from 4. on the dag.py workflow I click on the execution id
abcv46ptw8mxwgr5zv9r
it takes me back to screenshot 2 with the wrong workflow name!
Moving the same launch plan from one workflow to the other isn't updating the name of the workflow on the UI, correct? Are you just modifying the workflow in the launch plan?
> so it may be possible to work out which workflow to execute The other way seems more reasonable 'cause the focus has to be on the workflow, but not the launch plan. Having both ways can just be confusing for the users.
g

Gauthier Castro

11/09/2023, 1:28 PM
Moving the same launch plan from one workflow to the other isn’t updating the name of the workflow on the UI, correct? Are you just modifying the workflow in the launch plan?
Indeed, I’m changing the workflow in the launch plan from workflow_1.py to workflow_2.py. Then,
pyflyte register launch_plan.py
picks up all tasks, workflow and lp and registers a new version
s

Samhita Alla

11/09/2023, 1:45 PM
This is a bug. Can you create an issue? I don't think it's a good practice to use the same launch plan for two different workflows. So say, you want to create a launch plan for workflow 2, please create a new launch plan, but don't update the existing one.
g

Gauthier Castro

11/09/2023, 1:47 PM
So the use case here would be moving the launch plan to a new workflow rather than alternating 2 workflows. I’ve got a tight deadline before I go on leave tomorrow but will try to open an issue
btw, is this CLI docs wrong? https://docs.flyte.org/projects/flytekit/en/latest/pyflyte.html#cmdoption-pyflyte-package-o Shouldn’t it say something like “output tar file”?
s

Samhita Alla

11/09/2023, 1:50 PM
You're right.
If you could create an issue for this as well, that'd be great! 🙂
g

Gauthier Castro

11/09/2023, 1:55 PM
May have a 3rd one 😄 I’m using the
--source
flag on pyflyte package and although it works for the python module, pyflyte doesn’t find a json I import in my
workflow.py
’s code. But it works if I change the path to the json in the code so that it’s relative to where I’m running
pyflyte package
from, Probs more of a python thing that pyflyte!
I’ll definitely open issues, not 100% about the
when
!
Just getting back to this @Samhita Alla here is the issue for the UI potential bug https://github.com/flyteorg/flyte/issues/4575
2 Views