I've been testing the ContainerTask task type, and I would like to be able to pass a file in to a task that runs on a docker container (separate from the main) and get a file in return. We have lots of models we'd like to co-ordinate and each model is dockerised so that we can encapsulate them, and all of them expect to take a file as input and output to a file. However, with a minimal example — https://pastebin.com/PPMbs1uL — reading in a text file that contains the string "Hello World!", reading it and replacing "Hello" with "Goodbye" I end up with an empty file. To debug this, I set the "remove" kwargs to False from the run call in container_task.py#L280. In the docker logs I get:

cat: /var/inputs/input_file: No such file or directory

... which makes sense, the file isn't getting into the docker container. I look through the container_task.py source, and I can see that we bind a mount for the output, but I can't see how the inputs get in to the container, I had assumed that we'd mount two directories, one for inputs and one for outputs. How do inputs get into the ContainerTask? Can a FlyteFile be given as an input to a container?

Have you enabled copilot

Which copilot do you mean? There are so many products called copilot I lose track of which one is canonical. As for getting setup correctly, AFAICT, I've attached a debugger to script that runs and I can see all the steps that run. I've been digging more through the source, and I think I've got to pass in a file path to trigger the volume mount correctly in https://github.com/flyteorg/flytekit/blob/5012ded0feb7941ef3429c20f4c593452b4a8fe9/flytekit/core/container_task.py#L150 One thing I have noticed is that the regex to parse the input token —

r"^\{\{\s*\.inputs\.(.*?)\s*\}\}$"

places a very particular restriction that input values must be the only thing in the command element because of the

^

and

$

anchors which requires you to write your commands like:

command=[
        "/bin/sh", "-c",
        '"',
        "echo",
        "'",
        "{{.inputs.input_string}}",
        "'",
        " | sed -r 's/Hello/Goodbye/g' > /var/outputs/output_file",
        '"'
    ],

... rather than:

command=[
    "/bin/sh -c echo '{{.inputs.input_string}}' | sed -r 's/Hello/Goodbye/g' > /var/outputs/output_file"
],

... I'll probably raise a pull request to see if we can relax this requirement.