https://flyte.org logo
#ask-the-community
Title
# ask-the-community
n

Nuno Martins

06/13/2023, 9:23 AM
Hello, I am trying to launch a Flyte workflow to run remotely on a specific image (
pyflyte run
using the flags
--remote
and
--image <IMAGE>
), but I always get an error saying that I am missing packages, for example scikit-learn, which I indeed am missing locally. How can I get around this error, since the workflows will not be running locally? Thanks
a

Albert Wibowo

06/13/2023, 9:37 AM
I think you will have to create a custom docker image using dockerfile. You will have to specify your requirements.
n

Nuno Martins

06/13/2023, 9:39 AM
Do you mean the image i pass with the
--image
flag? The one I am passing has all the requirements
a

Albert Wibowo

06/13/2023, 9:39 AM
Yeah, that's how I did it last time.
n

Nuno Martins

06/13/2023, 9:41 AM
For me it is not working, it always complains that I am missing those dependencies locally, which I am since I don't intend to run them locally, only on the target docker image
j

Jan Fiedler

06/13/2023, 9:43 AM
pyflyte run still “compiles” your workflow and tasks locally before registering it to flyte, so i think for pyflyte run you need the requirements locally as well
n

Nuno Martins

06/13/2023, 9:44 AM
Is there a workaround for not compiling the workflow locally?
So I don't need the dependencies locally?
j

Jan Fiedler

06/13/2023, 9:46 AM
Hm i never tried but maybe switching to a dynamic workflow helps? Dynamic workflows are getting compiled during run time!
n

Nuno Martins

06/13/2023, 9:57 AM
Don't think that helps, still get the same error
s

Samhita Alla

06/14/2023, 4:36 AM
Can you try importing dependencies within the relevant tasks, if that's a possibility?
n

Nuno Martins

06/14/2023, 2:08 PM
If I move the imports inside the tasks then yes, I am able to do it. But I'd rather avoid it since importing other files from my project from inside a task would lead them not to be available at runtime when using "pyflyte run"
s

Samhita Alla

06/15/2023, 8:41 AM
In that case, you can use
pyflyte register
to register the whole directory that contains the relevant files.
2 Views