Hello, I am trying to launch a Flyte workflow to r...
# ask-the-community
n
Hello, I am trying to launch a Flyte workflow to run remotely on a specific image (
pyflyte run
using the flags
--remote
and
--image <IMAGE>
), but I always get an error saying that I am missing packages, for example scikit-learn, which I indeed am missing locally. How can I get around this error, since the workflows will not be running locally? Thanks
a
I think you will have to create a custom docker image using dockerfile. You will have to specify your requirements.
n
Do you mean the image i pass with the
--image
flag? The one I am passing has all the requirements
a
Yeah, that's how I did it last time.
n
For me it is not working, it always complains that I am missing those dependencies locally, which I am since I don't intend to run them locally, only on the target docker image
j
pyflyte run still “compiles” your workflow and tasks locally before registering it to flyte, so i think for pyflyte run you need the requirements locally as well
n
Is there a workaround for not compiling the workflow locally?
So I don't need the dependencies locally?
j
Hm i never tried but maybe switching to a dynamic workflow helps? Dynamic workflows are getting compiled during run time!
n
Don't think that helps, still get the same error
s
Can you try importing dependencies within the relevant tasks, if that's a possibility?
n
If I move the imports inside the tasks then yes, I am able to do it. But I'd rather avoid it since importing other files from my project from inside a task would lead them not to be available at runtime when using "pyflyte run"
s
In that case, you can use
pyflyte register
to register the whole directory that contains the relevant files.
109 Views