Hi, I am trying to save a directory to gcs using this code:

# save all results to gcs
    remote_dir = os.path.join(f'gs://{DATASETS_BUCKET}', '/path/to/output')

    return FlyteDirectory(path='/path/to/output', remote_directory=remote_dir)

but I am getting this error:

Failed to convert outputs of task 'flyte.workflows.wf.task1' at position 0:
  Failed to put data from /path/to/output/ to /path/to/output/ (recursive=True).

Original exception: '/path/to/output/htmls/2024/01/14/20240114114757_1a26bf7e-21b3-4e39-84a8-6bf02dff5bc3.txt' and '/path/to/output/htmls/2024/01/14/20240114114757_1a26bf7e-21b3-4e39-84a8-6bf02dff5bc3.txt' are the same file

what could the problem be? what is "same file" ? thanks

I think paths in your flyte directory is same but from the code snippet it doesn’t look like it

turns out that

os.path.join

does this: If a segment is an absolute path, then all previous segments are ignored and joining continues from the absolute path segment. and my

/path/to/output

is an absolute path, so it ignores the first gs part

@Nizar Hattab so is it working now?

yes