https://flyte.org logo
a

Ahmad Islam

04/28/2022, 6:50 AM
Hi there. I have Docker containers deployed on GKE. They got deployed and starts running at once. Output of one is based as input of next incoming container. Do we have such type of support in flyte so that it allows first one to finish and then let the next in the row to perform its job?
s

Samhita Alla

04/29/2022, 5:24 AM
Sorry, we missed your message. Could you elaborate on your use case? Do you mean to say you want to chain the containers?
a

Ahmad Islam

04/29/2022, 5:26 AM
No problem. Thanks again for response. Yes, I am talking about pipeline. Having multiple containers and output of first will be given to the next as input and so on. I don't want all the containers to run at once as It should wait for an input from subsequent one.
s

Samhita Alla

04/29/2022, 5:32 AM
Ok. You can easily achieve that through Flyte. If you’re okay with using a single image for your workflow, then chaining the `@task`s through a
@workflow
should work. But if you want to use multiple images, then the example I’ve sent should help you in doing so.
👍 1
a

Ahmad Islam

04/29/2022, 5:33 AM
Okay let me give it a try.
s

Samhita Alla

04/29/2022, 5:35 AM
The order of execution of tasks is determined by the input-output chaining. If a task’s output is to be used by a different task, then the order is determined as such, and the second task runs only after the first task’s run is complete. But if you do not have parameters to determine the order, you can enforce it using
>>
operator: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/chain_tasks.html.
🙏 1
a

Ahmad Islam

04/29/2022, 9:50 AM
Is that workflow similar for GKE or have to care about some additional dependencies?
s

Samhita Alla

04/29/2022, 9:58 AM
The workflow should work on GKE. You may have to install gsutil in your docker image: https://github.com/flyteorg/flytesnacks/blob/e9862d144d2e490269b4f9ed431dad802e9459a2/cookbook/core/Dockerfile#L17-L19.
1
6 Views