https://flyte.org
Join Slack
Is there a way to use conditionals or some other feature to end a workflow early based on a conditio...
# flyte-support
b
Is there a way to use conditionals or some other feature to end a workflow early based on a condition. For example, if I have a workflow that processes data and then trains a model, but have a parameter 
preprocess_only
, for example, is there a way to end after those tasks, and skip the rest, other than putting a conditional around all tasks after processing?
Or, a way to run several tasks inside one conditional, returning all of the values from the tasks, something like:
Copy code
train_metrics, eval_metrics, deploy_endpoint = conditional("preprocess_only").if_(preprocess_only.is_false()).then(
    train()
    evaluate()
    deploy()
)
t
how about you call a subworkflow within a conditional? that subworkflow can include the train, evaluate and deploy tasks. makes sense?
b
That does make sense. I’ll play around with that, I’m not familiar with how subworkflows show up in the UI
@tall-lock-23197 I think that will work well! Quick follow-up. Is there a way to rename the “branchNode” label, to know what’s inside without having to open each one?
t
would you mind sharing your code here? redacted should do.
b
for sure. here’s those two task calls, with conditionals
Copy code
pipeline_status_id = (
        conditional("create_status")
        .if_(production_run.is_true())
        .then(
            create_pipeline_status(...)
        )
        .else_()
        .then(noop())
    )

    dataset_bucket_dir = (
        conditional("dataset_bucket_dir")
        .if_(dataset_bucket_dir == "")
        .then(
            get_dataset_bucket_dir(...)
        )
        .else_()
        .then(return_str(value=dataset_bucket_dir))
    )
And the tasks are just decorated with 
@task(container_image=ImageSpec(...))
This is my workaround right now for our use of conditionals. There may be a better way to handle it, I’m not sure, but we just have several tasks that we only want to run conditionally, depending on which parts of the pipeline we want to run
t
This is my workaround right now for our use of conditionals.
i think subworkflow is the best technique. regarding conditionals, i think the latest version shouldn't display "branchNode". which version of flyte are you using?
b
Sounds good. I have started working in some subworkflows to clean some things up. I’ll continue with that and see if I can get it all into a better state. And I’m on 
1.10.2
right now
for 
flytekit
t
what about flyte?
b
My bad. Is it this? Same version? Or how do I check that one?
t
oh, should be the helm chart version you installed.
i think it's the same. is it flyte-core or flyte-binary helm chart?
b
I deployed using this: https://github.com/davidmirror-ops/davidmirror-flyte Which I think is flyte-binary? But I’m not positive. And it looks like it may be 1.10: 
version: v0.1.10 # VERSION
Oh wait, no its under flyte-core looks like
The commit sha that I’m on for that repo is 
db3132ac910ddb8c68a643990ddf10eafb6163d3
if that helps
t
have you used flyte-the-hard-way guide?
if so, it's flyte-binary.
b
Shoot, I sent the wrong repo, its actually this one: https://github.com/unionai-oss/deploy-flyte, and I’m on GCP, so I used this guide: https://github.com/unionai-oss/deploy-flyte/blob/main/environments/gcp/flyte-core/README.md along with some back and forth with David because some things weren’t quite working yet when I first tried it
And that’s under a 
flyte-core
directory, so that’s where I pulled that from. Sorry for the confusion
t
sorry. looks like it doesn't have to do with the latest version. i'll get back to you.
b
all good, thanks!
Hey, just wanted to check back in and see if anything was discovered here. Thanks!
t
hey, sorry i didn't get back to you earlier. this will be fixed in open source real soon!
b
Sounds great, thank you!
17 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by