Hi there. I have Docker containers deployed on GKE...
# announcements
a
Hi there. I have Docker containers deployed on GKE. They got deployed and starts running at once. Output of one is based as input of next incoming container. Do we have such type of support in flyte so that it allows first one to finish and then let the next in the row to perform its job?
s
Sorry, we missed your message. Could you elaborate on your use case? Do you mean to say you want to chain the containers?
a
No problem. Thanks again for response. Yes, I am talking about pipeline. Having multiple containers and output of first will be given to the next as input and so on. I don't want all the containers to run at once as It should wait for an input from subsequent one.
s
Ok. You can easily achieve that through Flyte. If you’re okay with using a single image for your workflow, then chaining the `@task`s through a
@workflow
should work. But if you want to use multiple images, then the example I’ve sent should help you in doing so.
👍 1
a
Okay let me give it a try.
s
The order of execution of tasks is determined by the input-output chaining. If a task’s output is to be used by a different task, then the order is determined as such, and the second task runs only after the first task’s run is complete. But if you do not have parameters to determine the order, you can enforce it using
>>
operator: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/chain_tasks.html.
🙏 1
a
Is that workflow similar for GKE or have to care about some additional dependencies?
s
The workflow should work on GKE. You may have to install gsutil in your docker image: https://github.com/flyteorg/flytesnacks/blob/e9862d144d2e490269b4f9ed431dad802e9459a2/cookbook/core/Dockerfile#L17-L19.
1
133 Views