New RunLLM response generated: >>>*# Question* Longtime listener, first time caller. I am trying to use the containertask with a list of flytefile like this:

calculate_ellipse_area_shell = ContainerTask(
   name="ellipse-area-metadata-shell",
   input_data_dir="/root/inputs",
   output_data_dir="/root/outputs",
   inputs=kwtypes(a=List[FlyteFile]),
   outputs=kwtypes(metadata=str),
   image="harbor.rke2/work:latest",
   command=[
       "/root/entrypoint.sh"
   ]
)

The container starts fine but can’t download the inputs and it looks like it cannot create the proper subdirectory that is named after the input variable name

time="2024-08-14T16:41:49Z" level=info msg="[0] Couldn't find a config file []. Relying on env vars and pflags."
2024-08-14T16:41:49.653575850Z {"json":{},"level":"error","msg":"Failed to persist [a], err failed to open file at path /root/inputs/a/0: open /root/inputs/a/0: no such file or directory","ts":"2024-08-14T16:41:49Z"}
{"json":{},"level":"error","msg":"Downloading failed, err failed to download input variable from remote store: variable [a] download/store failed: failed to open file at path /root/inputs/a/0: open /root/inputs/a/0: no such file or directory","ts":"2024-08-14T16:41:49Z"}

Anyone else using lists of flytefiles with the ContainerTask? # Answer Using a

ContainerTask

with a list of

FlyteFile

inputs is currently not supported by Flyte's CoPilot. The CoPilot component only supports primitive types and single

FlyteFile

or

FlyteDirectory

types, but not collections like

List[FlyteFile]

or

List[FlyteDirectory]

. According to this discussion, the CoPilot downloader does not support collection types, which is why you are encountering the error. The error message indicates that the CoPilot is unable to create the necessary subdirectory for the list of files. As a workaround, you can either: 1. Use individual

FlyteFile

inputs instead of a list. 2. Use a

FlyteDirectory

if all files are in a single directory. 3. Implement a custom task that handles the downloading of files from a list of URIs. Would you like more details on any of these workarounds? Sources: - Flyte Community Discussion Slack Link https://flyte-org.slack.com/archives/CP2HDHKE1/p1723655154.574839 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.