Question regarding fast registration and tar archives I ve n

Question

Question regarding fast registration and tar archives I ve noticed you recently added <https github com flyteorg flytekit blob master flytekit tools script mode py L77 L99|this> `tar strip file attributes` functions What is the purpose of that When unpacking fast registered archives it messes up the permissions ```d 3 root root 4096 Jul 14 14 04 unpacked rw r r 1 root root 335 Jul 14 13 59 scriptmode tar gz``` It doesn t matter if you run as root but we deliberately don t in our containers so everything fast registered with new flytekit currently breaks

Ketan (kumare3) · Answer

This is to get the same archive hash specifically the time component screws up the hash

Ketan (kumare3) · Answer

I think it s ok to keep user etc if you want

Tim Bauer · Answer

Ah damn makes sense

Ketan (kumare3) · Answer

Ya it was hard to figure that out

Tim Bauer · Answer

user is not the problem if unpacked by user that one owns it But we need `drwxr xr x`

Ketan (kumare3) · Answer

Ya I guess let s make pervasive perms

Ketan (kumare3) · Answer

I think that is safe we were too aggressive

Tim Bauer · Answer

Removing those <https github com flyteorg flytekit blob master flytekit tools script mode py L86 L87|two lines> results in those permissions when unpacked they work for non root user ```drwx 7 tim tim 4096 Jan 1 1970 fixed rw rw r 1 tim tim 583680 Jul 14 16 48 fixed tar gz``` uid and gid seem to overwritten when unpacking

Tim Bauer · Answer

This fixed fast registration again for us But `pyflyte run remote` still doesn t work I noticed that it sets the ` dest dir` argument to ` root` And also it seems to strip the workflow path defined in `flyte config` ``` sdk This option specifies the python packages where in to search for workflows and tasks workflow packages workflow packages=src workflows``` from the workflow files I m running a workflow called my wf from `src workflows examples hello world py` with pyflyte run remote this gets registered as examples hello world my wf the final argument to pyflyte execute thus also becomes ` task module examples hello world` but I want it to be ` task module src workflows examples hello world`

Ketan (kumare3) · Answer

You have missing init files

Ketan (kumare3) · Answer

We have decided to standardize names using init

Tim Bauer · Answer

Indeed thanks for quick spotting

Tim Bauer · Answer

One more thing I now noticed that pyflyte run doesnt pick up on the service account defined in flyte config ``` auth kubernetes service account=vault``` It runs pods with default

Tim Bauer · Answer

Thank you for always being so fast < Ketan kumare3 > btw I really wouldn t expect it from someone who is suppose is very busy but I get really thankful every time pray

Ketan (kumare3) · Answer

Ya the confit has been optimized

Ketan (kumare3) · Answer

We wanted to get rid of the config

Ketan (kumare3) · Answer

This might be an oversight but please help with this migration

Tim Bauer · Answer

Ok so there is no config anymore Can you point me to where <https github com flyteorg flytekit blob master flytekit clis sdk in container run py L525|this> `run level params` object is being put together in the click context

Tim Bauer · Answer

Ok I figured how to set dest dir and service account as arguments to pyflyte run Opened PR to remove the file permission reset <https github com flyteorg flytekit pull 1103>

Ketan (kumare3) · Answer

I delayed in responding cc < Eduardo Apolinario eapolinario >

Greg Gydush · Answer

gt One more thing I now noticed that pyflyte run doesnt pick up on the service account < Ketan kumare3 > having a similar issue where the service account specified during registration matches the service account used when a workflow is ran from the UI but does not match the service account used when executing from FlyteRemote

Greg Gydush · Answer

It looks like this problem started when we transitioned from flytekit 0 31 0 gt 1 0 3 still digging into it