Hello! We're deploying Flyte on EKS and prefer to use workflow-specific IAM roles. Could you say if there is a naming schema for writing temporary files to S3 we could depend on to restrict permissions at least on the project level? For example we see that currently there are multiple different naming, which makes it complicated to create reliable bucket access policies:

s3://.../2y/project-x/development/
s3://.../metadata/propeller/project-x-development-aq9kwx7nbqmxhccp2wgj/

can you do an

ls

on the contents of the first one please @Andrew Korzhuev

It looks like this:

2022-10-12 16:53:28      30720 0p/project-x/development/QMAM74EECKTJ4JDNWMCVXHEAH4======/fast91686145481555fce72a2a47137a26f2.tar.gz
2022-10-06 10:16:26      30720 0y/project-x/development/PB73FGHUK7JKFQRHCTKEKHT4AA======/fast2e6442b4f38ab209cae9f0b3292d1b57.tar.gz
2022-10-21 17:38:27      40960 0z/project-x/development/ZDWLO6645EE73ENUB4I2U4C764======/fastdb6e0c43aec1ae7b60cdd14797da2f54.tar.gz
2022-09-30 14:13:35      30720 1l/project-x/development/TVHEJEB5H3KG2FRKQ43IYHCKJI======/fast0400dadb6f2dac21d77422b62926eb19.tar.gz
2022-09-30 12:58:44      30720 1t/project-x/development/U7OQA2SO75O3VNX572PZ2XZH74======/fast3ebee174a68065b4841c2308f407c2e6.tar.gz
....
2022-09-14 10:08:15       1007 metadata/admin/project-x/development/.flytegen.workflow.say_hello/v1
2022-09-21 10:39:07       1010 metadata/admin/project-x/development/.flytegen.workflow.say_hello/v11
2022-09-21 16:30:01       1110 metadata/admin/project-x/development/.flytegen.workflows.example.say_hello/v10
2022-09-14 15:31:55        950 metadata/admin/project-x/development/flyte.workflows.example.my_wf/v1
2022-10-07 12:50:10       1749 metadata/admin/project-x/development/workflow.geolocation_wf/00d79fa06eb451f6e5f8747efb6bb3fdfce01de1
2022-10-28 15:03:52       1017 metadata/admin/project-x/development/workflow.geolocation_wf/0d6d35f
2022-10-21 17:38:28       7285 metadata/admin/project-x/development/workflow.geolocation_wf/18e0991
2022-10-19 16:49:50       4625 metadata/admin/project-x/development/workflow.geolocation_wf/25772cb
2022-10-11 11:30:55       1929 metadata/admin/project-x/development/workflow.geolocation_wf/6174a5b
....
2022-09-21 13:07:07          0 metadata/project-x/development/a2lg56hwv25rbxbg5v4x/inputs
2022-09-21 13:07:07          0 metadata/project-x/development/a2lg56hwv25rbxbg5v4x/user_inputs
2022-10-06 17:30:02          0 metadata/project-x/development/a2qqvz6lbcdd6pmngzwd/inputs
2022-10-06 17:30:02          0 metadata/project-x/development/a2qqvz6lbcdd6pmngzwd/user_inputs
2022-10-07 15:43:29          0 metadata/project-x/development/a46h5vpgv8zrbb5lf8xv/inputs
2022-10-07 15:43:29          0 metadata/project-x/development/a46h5vpgv8zrbb5lf8xv/user_inputs
2022-10-17 09:24:16          0 metadata/project-x/development/a4bqd77f8zr54vdlhm9d/inputs
2022-10-17 09:24:16          0 metadata/project-x/development/a4bqd77f8zr54vdlhm9d/user_inputs....
2022-09-21 14:46:05          0 metadata/propeller/project-x-development-a2lg56hwv25rbxbg5v4x/start-node/data/0/outputs.pb
2022-10-06 17:30:07          0 metadata/propeller/project-x-development-a2qqvz6lbcdd6pmngzwd/n0/data/inputs.pb
2022-10-06 17:30:07          0 metadata/propeller/project-x-development-a2qqvz6lbcdd6pmngzwd/start-node/data/0/outputs.pb
2022-10-07 15:43:59       2234 metadata/propeller/project-x-development-a46h5vpgv8zrbb5lf8xv/n0/data/0/error.pb
2022-10-07 15:43:34          0 metadata/propeller/project-x-development-a46h5vpgv8zrbb5lf8xv/n0/data/inputs.pb
2022-10-07 15:43:34          0 metadata/propeller/project-x-development-a46h5vpgv8zrbb5lf8xv/start-node/data/0/outputs.pb
2022-10-17 09:26:54       1335 metadata/propeller/project-x-development-a4bqd77f8zr54vdlhm9d/n0/data/0/error.pb
...

sorry can you clarify the ls please?

Under this dir I only see one file

2022-10-27 17:05:00      30720 2y/project-x/production/73IT45V7GAQCQ3SJAVSZPT2I5E======/fasta21ee57b163bb75f71419f3be22de564.tar.gz

Hi Andrew, following up on this just in case: the PR was merged and released in flyteadmin v1.1.55 here: https://github.com/flyteorg/flyteadmin/releases/tag/v1.1.55

Great, thank you!