Question regarding fast-registration and tar archi...
# flytekit
Question regarding fast-registration and tar archives: I've noticed you recently added this
functions. What is the purpose of that? When unpacking fast-registered archives it messes up the permissions:
Copy code
d--------- 3 root root 4096 Jul 14 14:04 unpacked
-rw-r--r-- 1 root root  335 Jul 14 13:59 scriptmode.tar.gz
It doesn't matter if you run as root but we deliberately don't in our containers so everything fast-registered with new flytekit currently breaks.
This is to get the same archive hash, specifically the time component screws up the hash
I think it's ok to keep user etc if you want
Ah damn makes sense
Ya it was hard to figure that out
user is not the problem, if unpacked by user that one owns it. But we need
Ya I guess let's make pervasive perms?
I think that is safe, we were too aggressive
Removing those two lines results in those permissions when unpacked: - they work for non root user
Copy code
drwx------ 7 tim tim   4096 Jan  1  1970 fixed
-rw-rw-r-- 1 tim tim 583680 Jul 14 16:48 fixed.tar.gz
uid and gid seem to overwritten when unpacking
This fixed fast-registration again for us. But
pyflyte run --remote
still doesn't work. I noticed that it sets the
argument to
And also it seems to strip the workflow path defined in `flyte.config`:
Copy code
# This option specifies the python packages where-in to search for workflows and tasks workflow packages.
from the workflow files. I'm running a workflow called "my_wf" from
- with pyflyte run --remote this gets registered as "examples.hello_world.my_wf", the final argument to pyflyte-execute thus also becomes
-- task-module examples.hello_world
but I want it to be
-- task-module src.workflows.examples.hello_world
You have missing init files
👍 1
We have decided to standardize names using init
Indeed, thanks for quick spotting
One more thing I now noticed that pyflyte run doesnt pick up on the service account defined in flyte.config:
Copy code
It runs pods with default
Thank you for always being so fast @Ketan (kumare3) btw. I really wouldn't expect it from someone who is suppose is very busy, but I get really thankful every time 🙏
Ya the confit has been optimized
We wanted to get rid of the config
This might be an oversight but please help with this migration
Ok so there is no config anymore? Can you point me to where this
object is being put together in the click context?
Ok I figured how to set dest-dir and service account as arguments to pyflyte run. Opened PR to remove the file permission reset:
I delayed in responding- cc @Eduardo Apolinario (eapolinario)
One more thing I now noticed that pyflyte run doesnt pick up on the service account
@Ketan (kumare3) having a similar issue where the service account specified during registration matches the service account used when a workflow is ran from the UI but does not match the service account used when executing from FlyteRemote
It looks like this problem started when we transitioned from flytekit 0.31.0 -> 1.0.3, still digging into it