Hello! I'm using

pyflyte run --remote ...

and everything is working, but I notice that my

.flyteignore

file is not being obeyed in

--copy auto

mode, only

all

. Is this the intended behavior? The function that lists the imported files isn't being passed the ignore_group object.

this was the intended behavior yeah, it was assumed perhaps incorrectly that if you loaded a python file, that it would need to be included. because if not, presumably when you go to run the task, the container will attempt to load it again. and if it doesn’t find it it’ll fail

Yeah, I'm testing things that way and by grabbing and inspecting the uploaded fast-serialized file from cloud storage. The specific issue is that my team's virtual environment folder is store directly alongside the project code, so that whole folder is packaged with

--copy auto

, typically something we've relied on ignore files to deal with. And

-v

makes is very obvious when a bunch of the file trees start with

.venv/...

. Unfortunately, all the pre-existing project tooling relies on the virtual environment being there, so moving it would be a whole project itself.

but that should already be taken care of https://github.com/flyteorg/flytekit/blob/3475ddc41f2ba31d23dd072362be704d7c2470a0/flytekit/tools/script_mode.py#L212-L214

I think I've found that

site.getsitepackages()

is returning a list of paths that combine to a

os.path.commonpath

that does not exist in the site packages list, so

if os.path.commonpath(site_packages + [mod_file]) in site_packages_set

is always false.

got it. would you mind making a pr out of this?

Yes, I can.

thank you thank you

thank you. if you can construct a unit test without too much time that would be much appreciated

According to the test I've written, it seems like my bugfix isn't working correctly in Windows. I may be able to get a proper Windows development environment running in several hours so I can debug it properly, instead of waiting for the CI to fail.