Has anyone done `FlyteDirectory` inputs to a `Cont...
# flyte-support
m
Has anyone done
FlyteDirectory
inputs to a
ContainerTask
? It seems like the download step doesn't correctly handle directories? with
inputs=dict(path=FlyteDirectory)
we're getting
Failed to download from ref [<gs://cradle-bio-pipelines/i9/f7aac4450113a425b905-n0-0/path>]
where that path does exist.
Following up on this. Problem was that the task didn't have an output declared, so the sidecar container exited immediately without downloading the input.
f
Ohh this is a bug, we are tracking to fix this month
q
but the input is downloaded by the init container right? the sidecar is only for uploading outputs...?
@melodic-magician-71351 see https://github.com/flyteorg/flyte/issues/3254 for the early sidecar exit... But I think this is not the cause of your problem, rather the sidecar shouldn't even be started if you don't have any output.
h
@melodic-magician-71351 and @quaint-diamond-37493 can you provide examples of the ContainerTask definition where these issues are occurring?
f
@hallowed-mouse-14616 I know
It’s a one line fix in the way we create the copilot job
I can create the PR or you can today
h
I have this on my TODO list for today, just easier to get test cases rather than write my own πŸ˜›
q
added a simple container tasks without outputs to the issue
πŸ™Œ 1
But it seems @melodic-magician-71351 has a different issue...
h
yeah, i agree. i'll take try to get them both wrapped up in the fix.
@melodic-magician-71351 / @quaint-diamond-37493 what version of k8s are you running? In testing here I'm seeing some interesting behavior with different versions that need to be sorted out.
q
1.24 or to be specific v1.24.6+k3s
πŸ‘ 1
m
Same
l
Hey @melodic-magician-71351, wondering if you could share your code for how you got this working with a FlyteDirectory input? I've tried it every way I can think of and nothing shows up in the container itself...
Copy code
bt = ContainerTask(
    name="basic-test",
    input_data_dir="/var/inputs",
    output_data_dir="/var/outputs",
    inputs=kwtypes(indir=FlyteDirectory),
    outputs=kwtypes(),
    image="<http://ghcr.io/flyteorg/rawcontainers-shell:v2|ghcr.io/flyteorg/rawcontainers-shell:v2>",
    command=[
        "ls",
        "-la",
        "/var/inputs",
    ],
)

@task
def get_dir(dirpath: str) -> FlyteDirectory:
    fd = FlyteDirectory(path=dirpath)
    return fd

@workflow
def wf():
    fd = get_dir(dirpath='<s3://my-s3-bucket/cv-in>')
    bt(indir=fd)
I also tried specifying outputs per your original follow-up, but that didn't help. I appreciate any insight you can provide!
m
Hey @loud-belgium-4006, I think we decided to go a different way with the workflow we needed this for. I'll double check with my team, but I'm fairly sure we never got this working.
l
Ah shame - oh well, thanks for following up!
159 Views