https://flyte.org logo
#ask-the-community
Title
# ask-the-community
n

Nizar Hattab

01/14/2024, 12:05 PM
Hi, I am trying to save a directory to gcs using this code:
Copy code
# save all results to gcs
    remote_dir = os.path.join(f'gs://{DATASETS_BUCKET}', '/path/to/output')

    return FlyteDirectory(path='/path/to/output', remote_directory=remote_dir)
but I am getting this error:
Copy code
Failed to convert outputs of task 'flyte.workflows.wf.task1' at position 0:
  Failed to put data from /path/to/output/ to /path/to/output/ (recursive=True).

Original exception: '/path/to/output/htmls/2024/01/14/20240114114757_1a26bf7e-21b3-4e39-84a8-6bf02dff5bc3.txt' and '/path/to/output/htmls/2024/01/14/20240114114757_1a26bf7e-21b3-4e39-84a8-6bf02dff5bc3.txt' are the same file
what could the problem be? what is "same file" ? thanks
j

Jay Ganbat

01/14/2024, 2:56 PM
I think paths in your flyte directory is same but from the code snippet it doesn’t look like it
Path and remote path are same
n

Nizar Hattab

01/15/2024, 9:20 AM
turns out that
os.path.join
does this: If a segment is an absolute path, then all previous segments are ignored and joining continues from the absolute path segment. and my
/path/to/output
is an absolute path, so it ignores the first gs part
d

David Espejo (he/him)

01/15/2024, 2:43 PM
@Nizar Hattab so is it working now?
n

Nizar Hattab

01/15/2024, 4:03 PM
yes
2 Views