Is there a way to use conditionals or some other feature to end a workflow early based on a condition. For example, if I have a workflow that processes data and then trains a model, but have a parameter

preprocess_only

, for example, is there a way to end after those tasks, and skip the rest, other than putting a conditional around all tasks after processing?

how about you call a subworkflow within a conditional? that subworkflow can include the train, evaluate and deploy tasks. makes sense?

That does make sense. I’ll play around with that, I’m not familiar with how subworkflows show up in the UI

would you mind sharing your code here? redacted should do.

for sure. here’s those two task calls, with conditionals

pipeline_status_id = (
        conditional("create_status")
        .if_(production_run.is_true())
        .then(
            create_pipeline_status(...)
        )
        .else_()
        .then(noop())
    )

    dataset_bucket_dir = (
        conditional("dataset_bucket_dir")
        .if_(dataset_bucket_dir == "")
        .then(
            get_dataset_bucket_dir(...)
        )
        .else_()
        .then(return_str(value=dataset_bucket_dir))
    )

And the tasks are just decorated with

@task(container_image=ImageSpec(...))

This is my workaround right now for our use of conditionals. There may be a better way to handle it, I’m not sure, but we just have several tasks that we only want to run conditionally, depending on which parts of the pipeline we want to run

This is my workaround right now for our use of conditionals.

i think subworkflow is the best technique. regarding conditionals, i think the latest version shouldn't display "branchNode". which version of flyte are you using?

Sounds good. I have started working in some subworkflows to clean some things up. I’ll continue with that and see if I can get it all into a better state. And I’m on

1.10.2

right now

what about flyte?

My bad. Is it this? Same version? Or how do I check that one?

oh, should be the helm chart version you installed.

I deployed using this: https://github.com/davidmirror-ops/davidmirror-flyte Which I think is flyte-binary? But I’m not positive. And it looks like it may be 1.10:

version: v0.1.10 # VERSION

have you used flyte-the-hard-way guide?

Shoot, I sent the wrong repo, its actually this one: https://github.com/unionai-oss/deploy-flyte, and I’m on GCP, so I used this guide: https://github.com/unionai-oss/deploy-flyte/blob/main/environments/gcp/flyte-core/README.md along with some back and forth with David because some things weren’t quite working yet when I first tried it

sorry. looks like it doesn't have to do with the latest version. i'll get back to you.

all good, thanks!

hey, sorry i didn't get back to you earlier. this will be fixed in open source real soon!

Sounds great, thank you!