:wave: Hi! pyflyte fast registration command takes...
# ask-the-community
v
đź‘‹ Hi! pyflyte fast registration command takes at least 5+ minutes every time when packaging and registering via github action on to a standalone cluster? Thoughts?
f
Can you please check your root folder size with
du -chd 1
?
(Ah, didn’t see this is in CICD, this makes it less likely the folder is large.) The contents of the root folder are copied into a tarball and uploaded to blob storage. I have seen in the past that e.g. the
.git
folder is large or that ML engineers stored data within the python package the workflow belongs to. This data is then uploaded as part of the tarball everytime during fast-registration.
v
Yep. I saw that even though I copy all the python code into the folder and pass that folder to the command line it falls back to the root/home directory. Hmm....
Yep. And that is huge, as it is the github runner homedir that checks out the whole repo which has other other ML model code that we don't want to package.
How do I tell it to not to that? 🙂
n
@Kevin Su pyflyte fast registration should avoid packaging files that are in
.gitignore
correct?
k
currently all the file in the root folder will be packaged.
n
huh, doesn’t this https://github.com/flyteorg/flytekit/pull/899 take care of ignoring files in
.gitignore
or
.dockerignore
? Is the root folder exempt from this?
k
sorry, I forgot this pr. yeah, .gitignore should work in pyflyte fast registration
k
it does, skip .gitignore and .dockerignore
n
great! created this issue since our current registration docs doesn’t cover this: https://github.com/flyteorg/flyte/issues/3407
v
Thanks All! That worked! Used the .dockerignore
ls -1a /home/runner | grep -wv app_module >> /home/runner/.dockerignore
k
we are actually thinking of working on a customizable package / run / register command so that you can choose the package root
150 Views