is there an easy way to mount some dir from execution space to container running task, or the only option is to fully build image, publish and then pull into k8s? I want fast iterations 🙂

Are you using pyflyte run

probably problem is that my code is not inside the top dir, but higher and the code is not packed

Hmm this is interesting

do you have

__init___.py

in the folder

mlrobot/mlrobot/workflows$ tree . . ├── init.py ├── example.py but actually mlrobot (the parent dir) also contains init.py and I use scripts from this module

which version of flytekit are you using

I guess it’s 1.5

is the class you import in example.py a flyte task or workflow?

ah no

if not, flytekit wont copy that module

it’s just external code, here is the tree of the parent dirs: mlrobot

├── mlrobot
│   ├── __init__.py
│   ├── extra
│   │   └── __init__.py
│   ├── workflows
│   │   ├── example.py

The idea behind this is that we don’t want to copy all the files in

source_root

. some people may have lot of unnecessary files in that folder. btw only flyte task and workflow has metadata container relative path of file

so is it like Flyte is deigned to work with existing packages/prepared images, not to add workflows to current-project code? it’s a bit misleading; so this --copy-all is yet to be implemented?

not yet, I can add that tomorrow. we encourage people to use non-fast register (image contains python package and code) when running workflow in production because it saves time by not having to copy code to the pod from S3

it makes sense for production, but for fast experiments it’s a bit overwhelming (we used to have luigi and now would like to have tasks distribution, so we try flyte for that purpose) thanks! would be great to have that feature

I see. pyflyte register is also fast-register (will copy the code to s3) by default. To workaround it, you could use pyflyte register + manually trigger workflow from UI

yeah, it’s the current solution, but we have really huge configs, usually in such systems it’s possible to trigger from UI with json description of the params, is it also possible with Flyte? (we have ~100 lines of configs for our regular pipelines with ~20 different params)

@Anatolii Bardukov Could you help me test it?

pip install git+<https://github.com/flyteorg/flytekit@769334528e99df774cae86a1f18de55b22130819>