Hello guys! Does gcs account need to have Delete ...
# ask-the-community
n
Hello guys! Does gcs account need to have Delete permissions on buckets to be used in flyte files operations ? getting this error when using
Copy code
FlyteDirectory(path='path/to/parent', remote_directory='<gs://datasets_bucket/path/to/parent>')
Copy code
[0]: code:"SYSTEM:Unknown" message:"Traceback (most recent call last):\n\n      File \"/opt/venv/lib/python3.9/site-packages/flytekit/exceptions/scopes.py\", line 165, in system_entry_point\n        return wrapped(*args, **kwargs)\n      File \"/opt/venv/lib/python3.9/site-packages/flytekit/core/base_task.py\", line 588, in dispatch_execute\n        raise TypeError(msg) from e\n\nMessage:\n\n    Failed to convert outputs of task 'workflows.wfname.map_extract_f6482939eed2867de4faf3d9d475fe60' at position 0:\n  Failed to put data from /tmp/flyte-_2xcubb6/sandbox/local_flytekit/d8544a75c07b4b0a894056feafcf0bee/path/to/parent to <gs://datasets_bucket/path/to/parent> (recursive=True).\n\nOriginal exception: Forbidden: <https://storage.googleapis.com/upload/storage/v1/b/datasets_bucket/o>\naccount_name does not have storage.objects.delete access to the Google Cloud Storage object.\n\nSYSTEM ERROR! Contact platform administrators." kind:SYSTEM 
... and many more.
why delete when it is trying to upload? and if the directory path/to/parent does not exist or empty, does it delete contents of gcs corresponding path ?
j
this is because of how composition works. composition is used for uploading large files to GCS. basically the file is chunked, chunks are uploaded individually, and composed back into its original form in GCS. the last step is to delete the temporary chunked files.
n
Thanks! that explains alot. @Ariel Kaspit