Has anyone done `FlyteDirectory` inputs to a `Cont...
# ask-the-community
e
Has anyone done
FlyteDirectory
inputs to a
ContainerTask
? It seems like the download step doesn't correctly handle directories? with
inputs=dict(path=FlyteDirectory)
we're getting
Failed to download from ref [<gs://cradle-bio-pipelines/i9/f7aac4450113a425b905-n0-0/path>]
where that path does exist.
Following up on this. Problem was that the task didn't have an output declared, so the sidecar container exited immediately without downloading the input.
k
Ohh this is a bug, we are tracking to fix this month
f
but the input is downloaded by the init container right? the sidecar is only for uploading outputs...?
@Eli Bixby see https://github.com/flyteorg/flyte/issues/3254 for the early sidecar exit... But I think this is not the cause of your problem, rather the sidecar shouldn't even be started if you don't have any output.
d
@Eli Bixby and @Felix Ruess can you provide examples of the ContainerTask definition where these issues are occurring?
k
@Dan Rammer (hamersaw) I know
It’s a one line fix in the way we create the copilot job
I can create the PR or you can today
d
I have this on my TODO list for today, just easier to get test cases rather than write my own 😛
f
added a simple container tasks without outputs to the issue
But it seems @Eli Bixby has a different issue...
d
yeah, i agree. i'll take try to get them both wrapped up in the fix.
@Eli Bixby / @Felix Ruess what version of k8s are you running? In testing here I'm seeing some interesting behavior with different versions that need to be sorted out.
f
1.24 or to be specific v1.24.6+k3s
e
Same
p
Hey @Eli Bixby, wondering if you could share your code for how you got this working with a FlyteDirectory input? I've tried it every way I can think of and nothing shows up in the container itself...
Copy code
bt = ContainerTask(
    name="basic-test",
    input_data_dir="/var/inputs",
    output_data_dir="/var/outputs",
    inputs=kwtypes(indir=FlyteDirectory),
    outputs=kwtypes(),
    image="<http://ghcr.io/flyteorg/rawcontainers-shell:v2|ghcr.io/flyteorg/rawcontainers-shell:v2>",
    command=[
        "ls",
        "-la",
        "/var/inputs",
    ],
)

@task
def get_dir(dirpath: str) -> FlyteDirectory:
    fd = FlyteDirectory(path=dirpath)
    return fd

@workflow
def wf():
    fd = get_dir(dirpath='<s3://my-s3-bucket/cv-in>')
    bt(indir=fd)
I also tried specifying outputs per your original follow-up, but that didn't help. I appreciate any insight you can provide!
e
Hey @Pryce, I think we decided to go a different way with the workflow we needed this for. I'll double check with my team, but I'm fairly sure we never got this working.
p
Ah shame - oh well, thanks for following up!