Ahmad Islam

    Ahmad Islam

    4 months ago
    Hi there. I have Docker containers deployed on GKE. They got deployed and starts running at once. Output of one is based as input of next incoming container. Do we have such type of support in flyte so that it allows first one to finish and then let the next in the row to perform its job?
    Samhita Alla

    Samhita Alla

    4 months ago
    Sorry, we missed your message. Could you elaborate on your use case? Do you mean to say you want to chain the containers?
    Ahmad Islam

    Ahmad Islam

    4 months ago
    No problem. Thanks again for response. Yes, I am talking about pipeline. Having multiple containers and output of first will be given to the next as input and so on. I don't want all the containers to run at once as It should wait for an input from subsequent one.
    Samhita Alla

    Samhita Alla

    4 months ago
    Ok. You can easily achieve that through Flyte. If you’re okay with using a single image for your workflow, then chaining the @tasks through a
    @workflow
    should work. But if you want to use multiple images, then the example I’ve sent should help you in doing so.
    Ahmad Islam

    Ahmad Islam

    4 months ago
    Okay let me give it a try.
    Samhita Alla

    Samhita Alla

    4 months ago
    The order of execution of tasks is determined by the input-output chaining. If a task’s output is to be used by a different task, then the order is determined as such, and the second task runs only after the first task’s run is complete. But if you do not have parameters to determine the order, you can enforce it using
    >>
    operator: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/chain_tasks.html.
    Ahmad Islam

    Ahmad Islam

    4 months ago
    Is that workflow similar for GKE or have to care about some additional dependencies?
    Samhita Alla

    Samhita Alla

    4 months ago
    The workflow should work on GKE. You may have to install gsutil in your docker image: https://github.com/flyteorg/flytesnacks/blob/e9862d144d2e490269b4f9ed431dad802e9459a2/cookbook/core/Dockerfile#L17-L19.