https://flyte.org logo
Join Slack
Powered by
Hiya, is it possible to have multiple tasks in a b...
# flyte-support
g
Hiya, is it possible to have multiple tasks in a branch of a conditional? All the docs point only show 1 task, but I'm interested if something like the following is possible:
Copy code
results = (
        conditional("if-flag")
        .if_(flag.is_true())
        .then((
              out1 = task1(input1),
              task2(out1)
           ))
        .else_().then(noop())
    )
Currently I pass a workflow that looks like this into my conditional and it works fine:
Copy code
@workflow
def my_sub_workflow(input1, input2):
    out1 = task1(input1)
    out2 = task2(input2)
    out3 = task3(out1, out2)

@workflow
def main_workflow(input1, flag):
    input2 = another_task(input1)
    (
        conditional("if-flag")
        .if_(flag.is_true())
        .then(my_sub_workflow(input1, input2, flag))
        .else_().then(noop())
    )
However as you can see 
task1
which I do want to execute conditionally with the rest of the tasks in the subworkflow depends only on 
input1
(which is available at the start of the execution). But 
task1
has to wait until the 
input2
is available after 
another_task
has run and the subworkflow has kicked off. 
task1
is also generally a longer running task than 
task2
so it would be beneficial to start it as soon as possible so that it completes by the time 
task2
is done. So A) is the former possible, and more importantly, B) Is there a better way to organize the tasks so that 
task1
can start earlier without having to wait for the subworkflow to start?
Already have tried rewriting it in a way that Flyte doesn't seem to like (it fails to compile with the message 
Cannot pass output from task n2 that produces no outputs to a downstream task
):
Copy code
noop = Echo(name="noop")

@workflow
def main_workflow(input1, flag):
    input2 = another_task(input1)
    out1 = (
        conditional("if-flag1")
        .if_(flag.is_true())
        .then(task1(input1))
        .else_().then(noop())
    )
    out2 = (
        conditional("if-flag2")
        .if_(flag.is_true())
        .then(task2(input2))
        .else_().then(noop())
    )
    out3 = (
        conditional("if-flag3")
        .if_(flag.is_true())
        .then(task3(out1, out2))
        .else_().then(noop())
    )
I understnd that that is due to that there is a chance (even if the flags for each conditional are the same...) 🤔 Perhaps if I update my 
noop
to return some value it would work
Hmm. No. That wouldn't work either because task2 returns a 
FlyteDirectory
and I'm not sure you can mock/Echo a 
FlyteDirectory
output, can you? EDIT: You can. Seems like I might have a way forward then, albeit ugly a bit ugly with more conditionals when really it is one conditional
To clarify, I need an Echo task per output type for each 
_else
branch. Then flyte will compile and run it.
Still curious if there is a neater way to write this instead of creating 3 conditionals with the same condition
c
We're investing in eager workflows, which will obviate the need for conditionals and other contorting alternatives in flyte. It'll take some time, but it's an active area of investigation, so expect results in the coming months.
g
Looking forward to it:)
8 Views
Open in Slack
PreviousNext
Powered by