Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Uh. Is it true that I can't use `FlyteDirectory` type inputs for `ContainerTask`s?

When I try to do this, I get:
```Currently only single part blobs are supported, we will force multipart to be 'path/00000'"
"Failed to download from ref [<gs://flyte-bucket/data/r5/apv2mgflkqlq9pp6rz2t-n1-0-n1-n0-0-n0-0/90aa7e15e7ec2c26f237238d38632a4a/000000>]"
"Failed to persist [input_dir], err incorrect blob reference, does not exist", "ts":"2024-03-18T16:01:45Z"
"Downloading failed, err failed to download input variable from remote store: variable [input_dir] download/store failed: incorrect blob reference, does not exist"```


And I'm guessing `FlyteDirectory`s are also not allowed as outputs of `ContainerTask`s

Seems like it: <https://github.com/flyteorg/flyte/issues/3632>

cc <@U05FFAF4FC2> we should support FlyteFile/FlyteDirectory in local execution as well

At least did not have issues with local FlyteDirectory inputs/outputs with python tasks. But I guess that's also true for remote

yes, but we can’t run `ContainerTask` locally for now

Ah, yeah. Would be great to test such workflows e2e locally too

<@U05FFAF4FC2> is working on it. <https://github.com/flyteorg/flytekit/pull/2258>

For now I'm thinking to make a workaround that where either the task preceding the container task outputs a
`List[FlyteFile]`  and then just map a task over this list and recombine the outputs. Still, being able to use `FlyteDirectory` directly and letting this mapping happen in the container (as the number of Files to be processed is small) would be more convenient

I think `List` and `Dict`  is not supported in Container task
even the code `flytecopilot`  seems implemented them well

<https://github.com/flyteorg/flytekit/pull/2258>

This PR can help you test `FlyteFile` and `FlyteDirectory`  's local execution

&gt;  I think `List` and `Dict`  is not supported in Container task
&gt; even the code `flytecopilot`  seems implemented them well
But individual `FlyteFile`s should be fine, right? like when a `List` of `FlyteFile` s is mapped using something like `map_task` or `array_node_map_task` right?

For map task, there is a PR supporting container task with map task

Wait. So that's also not supported yet? Hmm.

container task needs to implement lots of complex logic

Eventually circumvented my issue by creating an imagespec with the container as the base image and then creating a PythonTask that uses that imagespec to call the relevant applications in the image and handle inputs/outputs