Hi. I'm still struggling to launch my workflow from a container inside of Sandbox environment. My workflow requires

verstack

so I included it to requirements.txt. Here is the outline what I did:

$ pyflyte init my_first_wf

$ ./bin/flytectl sandbox start --source .

Then I wrote some codes under workflows directory. I tested locally using python command and it worked without any problem, cuz verstack is inside venv library. Now I built an image using bundled docker_build.sh like this:

$ sh docker_build.sh -r localhost:30000

$ pyflyte --pkgs <http://workflows.my|workflows.my>_workflow package --image  localhost:30000/my_first_wf:fbc629e54b46c12d6396d39ea9d3f37307ba961d

At this moment, when I launch a container at localhost3000/my first wffbc629e54b46c12d6396d39ea9d3f37307ba961d, the python environment recognizes verstack.

$  docker run -it localhost:30000/my_first_wf:fbc629e54b46c12d6396d39ea9d3f37307ba961d sh

# python

>>> import verstack

(no problem)

However, when I try to run my workflow inside the demo cluster, it fails because python couldn't find verstack library.

pyflyte run --remote workflows/my_workflow.py  train --filename '$VVIX.csv' --test_size 0.25 --output_path model.model

The import statement fails because it couldn't find verstack, and attached is a part of the log.

Pod failed. No message received from kubernetes.
[fd4d75a833c134b5f9e1-n0-0] terminated with exit code (1). Reason [Error]. Message: 
l>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/workflows/my_workflow.py", line 11, in <module>
    from verstack import LGBMTuner 
ModuleNotFoundError: No module named 'verstack'

Am I successfully launching a pod that I made, or am I launching a different version of pod?

you have to specify image in pyflyte command. pyflyte run --remote --image my_first_wf:fbc629e54b46c12d6396d39ea9d3f37307ba961d …

Thanks. I think I'm almost there, however, I got this.

containers with unready status: [f559893b4d3564dedad1-n0-0]|Back-off pulling image "localhost:30000/my_first_wf:fbc629e54b46c12d6396d39ea9d3f37307ba961d"

did you push the image to docker hub

I don't think so. I believe I pushed to localhost:30000

you have to push the image somewhere that k8s can find it. k3s is running in another docker container in demo cluster. which means we have docker container running in the docker container. therefore, k8s can’t find your image in the localhost. try to push the image to public registry, then k8s can pull the image

Hi @Masa Nakamura Did pushing your image to a public registry fixed your issue?

@David Espejo (he/him) Yes, now my workflow is running on local demo cluster.

great! thank you for confirming. Just let us know how else we can help 🙂

Unfortunately, my client wouldn't allow public repo, so I was looking for a solution. This worked for me:

$ sh docker_build.sh -p second -y 3.9.16 -r localhost:30000
$ docker push localhost:30000/second:<tag>
$ pyflyte run --remote --image   localhost:30000/second:<tag> workflows/my_workflow.py  <workflowname> --argname value ....