Hello, I am a software engineer and currently evaluating Flyte as a workflow backend for our media processing platform. I want to run a custom docker-image as task with runtime generated command-line based on task input. Is that possible? All docs only show how to tun a custom image with static (compile-time) command-line. I also tested

ContainerTask

wich can mount inputs into

input_data_dir

but that would require the workaround to write a shell script that reads the input and builds + runs the actual command-line.

@task(task_config=Pod(pod_spec=V1PodSpec(...)))
def pod_task(name: str):
    """how to inject task input `name` into args command-line for my custom docker image sidecar container?"""

I'm not sure if this is exactly what you're looking for but: https://github.com/flyteorg/flytesnacks/blob/master/examples/blast/blast/blastx_example.py#L50 seems somewhat similar to what you want to do. Alternatively, I guess you could use

subprocess.run

?

Hey @René Persé, if I'm understanding correctly, you'd like to run a specific custom image depending on what the task input is?

@Dan Farrell This will be a workaround but I like to avoid building new flyte-specific docker images for our existing tools. @Pryce & all: I want to use existing docker images, I maybe have no control in, inside my flyte workflow. But I want to use task input variables inside the docker image arguments. Example (not working, but I like to use something like

{inputs.message}

in my argument line):

@task(
    task_config=Pod(
        pod_spec=V1PodSpec(
            containers=[
                V1Container(
                    name="notify",
                    command=["cowsay"],
                    args=["{inputs.message}"],
                    image="docker/whalesay",
                )
            ]
        )
    )
)
def notify(message: str) -> str:
    ...

Hi @René Persé, it looks like you're overloading the pod_spec a bit. Using a

ContainerTask

will already override whatever is specified in the entrypoint/cmd of the Dockerfile. So it should look something like this:

notify = ContainerTask(
    name="notify",
    input_data_dir="/var/inputs",
    output_data_dir="/var/outputs",
    inputs=kwtypes(exec=float, args=float),
    outputs=kwtypes(out=str),
    image="ghcr.io/flyteorg/rawcontainers-shell:v2",
    command=[
        "{{.inputs.exec}}",
        "{{.inputs.args}}",
        "-o {{.outputs.out}}",
    ],
)

Perhaps if you could share more specifically what container, commands and expected inputs and outputs are I could assist more specifically. Hope it helps.