https://flyte.org
Join Slack
Hi comm, I'm new to Flyte. I have been trying to get around this, but don't know what the issue is...
# flyte-support
m
Hi comm, I'm new to Flyte. I have been trying to get around this, but don't know what the issue is. I installed the flyte sandbox to test things out. When I run a workflow locally, it works fine and completed (typically completed in 3 mins). However, when I run it remotely in the sandbox, it takes very long time (about an hour) and eventually fails with this error: 
Pod failed. No message received from kubernetes.
t
Welcome @mysterious-hair-40059 to the community. Have you run 
flytectl demo start
to spin up the cluster? And what workflow are you trying to run?
m
Yes I have
I created a small custom workflow. Although the quick one from the getting started page on the doc also had the same issue.
a
HI @mysterious-hair-40059 could you share the output of 
kubectl get pods -n flyte
t
Could you also share the output of 
kubcetl logs <pod-name> -n flytesnacks-development
? You can get the pod that's erroring out by running 
kubectl get pods -n flytesnacks-development
considering 
flytesnacks-development
is the namespace you triggered the workflow in.
m
Thanks guys @cold-lock-43986 @tall-lock-23197 This is the error I get on describing the pod in 
flytesnacks-development
namespace (screenshot) And when I check the logs, it says trying and failing to pull image.
t
I believe your system isn't allowing you to pull the image.
m
Thanks @tall-lock-23197 still didn't work. However, I'm already going deeper into registering workflows. I want to ask, how do I ship in all necessary packages that are needed to make a workflow run successfully? for example, I've modularized my code and the tasks arranged into a workflow are in different modules of the project, will Flyte be able to package all these when I run "pyflyte register", "pyflyte packages" or "flytctl register"? Also, how do I register workflows with different tasks from a Jupyter Notebook?
t
@mysterious-hair-40059, you can use 
pyflyte register
to package up all your tasks and workflows present in different modules. You can register workflows from a Jupyter notebook using 
register_workflow
method: https://github.com/flyteorg/flytekit/blob/707fc03b88ec08fb10d60ac642f3c80e275bd90c/tests/flytekit/unit/remote/test_sandbox.py#L70-L77.
✅ 1
m
Ok. Thank you very much.
Hello @tall-lock-23197 Can you help with this issue? https://flyte-org.slack.com/archives/CP2HDHKE1/p1676980150513129
c
HI all
I was trying to push into docker registry provided by sandbox but its gives me issue
pandas_image_spec = custom_image = ImageSpec(base_image="cr.flyte.org/flyteorg/flytekit:py3.8-1.10.2", requirements="requirements.txt", registry="flyte-sandbox-docker-registry:30000", )
#10 ERROR: failed to do request: Head "https://flyte-sandbox-docker-registry:30000/v2/flytekit/blobs/sha256:95c8face58787aec77a706785d937c9a79e27a10ad7160b14603794e6cc1f6e6": dial tcp: lookup flyte-sandbox-docker-registry on 192.168.65.7:53: no such host ------
exporting to image:
------ Failed with Unknown Exception <class 'Exception'> Reason: failed to run command envd build --path /var/folders/3t/zknn95217k1_glry5tdyxrzm0000gn/T/flytea9z7hopd/control_plane_metadata/local_flytekit/f0586d94ea435c9d66d7010b9688ba4b --platform linux/amd64 --output type=image,name=flyte-sandbox-docker-registry:30000/flytekit:FG_FuPzAsM0KRkvtHteClg..,push=true with error b'error: failed to do request: Head "https://flyte-sandbox-docker-registry:30000/v2/flytekit/blobs/sha256:95c8face58787aec77a706785d937c9a79e27a10ad7160b14603794e6cc1f6e6": dial tcp: lookup flyte-sandbox-docker-registry on 192.168.65.7:53: no such host\n' failed to run command envd build --path /var/folders/3t/zknn95217k1_glry5tdyxrzm0000gn/T/flytea9z7hopd/control_plane_metadata/local_flytekit/f0586d94ea435c9d66d7010b9688ba4b --platform linux/amd64 --output type=image,name=flyte-sandbox-docker-registry:30000/flytekit:FG_FuPzAsM0KRkvtHteClg..,push=true with error b'error: failed to do request: Head "https://flyte-sandbox-docker-registry:30000/v2/flytekit/blobs/sha256:95c8face58787aec77a706785d937c9a79e27a10ad7160b14603794e6cc1f6e6": dial tcp: lookup flyte-sandbox-docker-registry on 192.168.65.7:53: no such host\n'
380 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by