https://flyte.org logo
Join Slack
Powered by
Hello Team, I am running flyte on microk8s cluster...
# flyte-support
a
Hello Team, I am running flyte on microk8s cluster. Cluster itself is created on Google VM's I am trying to understand the error and how to solve it. Error:
Copy code
pyflyte register --project $PROJECT --domain development test/flyte/

MaxRetryError: HTTPConnectionPool(host='minio.flyte.svc.cluster.local', port=9000): Max retries exceeded with url:
/flyte/eddi-test/development/DEHEROM77DXITI3GJPBQF2QK7E%3D%3D%3D%3D%3D%3D/fastbdb56d92944baea095e237e4ba4513ee.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20250408%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250408T123413Z&X-Amz-Expires=3600&X-Amz-SignedHead
ers=content-md5%3Bhost%3Bx-amz-meta-flytecontentmd5&X-Amz-Signature=17372653d8e002f5364f34e43ace70cb7ede2fde5fc74f78f47e31d636af01ed (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7da6087a2080>: Failed to establish a new connection: [Errno -3]
Temporary failure in name resolution'))
I have done as per official documentation but unable to move forward
Copy code
k -n flyte port-forward service/minio 9000:9000

k -n flyte port-forward service/flyteadmin 8089:81

k -n flyte port-forward service/flyteconsole 8088:80
I have set some variables as below
Copy code
export FLYTE_PLATFORM_URL=localhost:8089
export FLYTE_PLATFORM_INSECURE=true
export FLYTE_BLOBSTORE_ENDPOINT=localhost:9000
export FLYTE_BLOBSTORE_INSECURE=true
This can be verified by running processes on flyte-master
Copy code
ps aux |grep port-forward
root        9455  0.0  0.0   8560  3584 pts/6    S+   11:49   0:00 bash /snap/microk8s/7964/microk8s.wrapper kubectl -n flyte port-forward service/minio 9000:9000
root        9472  0.0  0.0   8424  3584 pts/6    S+   11:49   0:00 bash /snap/microk8s/7964/microk8s-kubectl.wrapper -n flyte port-forward service/minio 9000:9000
root        9481  0.0  1.2 1283480 49812 pts/6   Sl+  11:49   0:00 /snap/microk8s/7964/kubectl -n flyte port-forward service/minio 9000:9000
root        9926  0.0  0.0   8560  3584 pts/8    S+   11:49   0:00 bash /snap/microk8s/7964/microk8s.wrapper kubectl -n flyte port-forward service/flyteconsole 8088:80
root        9942  0.0  0.0   8424  3712 pts/8    S+   11:49   0:00 bash /snap/microk8s/7964/microk8s-kubectl.wrapper -n flyte port-forward service/flyteconsole 8088:80
root        9951  0.0  1.2 1283736 48560 pts/8   Sl+  11:49   0:00 /snap/microk8s/7964/kubectl -n flyte port-forward service/flyteconsole 8088:80
root       30686  0.0  0.1   8560  4096 pts/7    S+   12:08   0:00 bash /snap/microk8s/7964/microk8s.wrapper kubectl -n flyte port-forward service/flyteadmin 8089:81
root       30702  0.0  0.1   8424  4096 pts/7    S+   12:08   0:00 bash /snap/microk8s/7964/microk8s-kubectl.wrapper -n flyte port-forward service/flyteadmin 8089:81
root       30711  0.0  1.2 1283736 49868 pts/7   Sl+  12:08   0:00 /snap/microk8s/7964/kubectl -n flyte port-forward service/flyteadmin 8089:81
root       74241  0.0  0.0   7008  2304 pts/1    S+   12:41   0:00 grep --color=auto port-forward
Cluster Nodes are as below. Not these are microk8s install cluster
Copy code
k get nodes -owide
NAME           STATUS   ROLES     AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
cmpn1          Ready    <none>    2d12h   v1.32.3   10.128.0.26   <none>        Ubuntu 22.04.5 LTS   6.8.0-1026-gcp   <containerd://1.6.36>
cmpn2          Ready    <none>    2d12h   v1.32.3   10.128.0.28   <none>        Ubuntu 22.04.5 LTS   6.8.0-1026-gcp   <containerd://1.6.36>
flyte-master   Ready    <none>    2d12h   v1.32.3   10.128.0.30   <none>        Ubuntu 22.04.5 LTS   6.8.0-1026-gcp   <containerd://1.6.36>
sto1           Ready    storage   2d12h   v1.32.3   10.128.0.29   <none>        Ubuntu 22.04.5 LTS   6.8.0-1026-gcp   <containerd://1.6.36>
What I have observed is , when I run pyflyte register, one port forward terminal returns error
Copy code
(base) root@flyte-master:~# k -n flyte port-forward service/flyteadmin 8089:81
Forwarding from 127.0.0.1:8089 -> 8089
Forwarding from [::1]:8089 -> 8089 
Handling connection for 8089
E0408 12:46:49.616454   79936 portforward.go:404] "Unhandled Error" err="error copying from local connection to remote stream: writeto tcp6 [::1]:8089->[::1]:45926: read tcp6 [::1]:8089->[::1]:45926: read: connection reset by peer
a
Maybe the local DNS resolution for 
localhost
is also pointing to the IPv6 loop back?
a
ANy idea how to check that? flyte is on microk8s cluster and pyflyte just runs on VM
a
writeto tcp6 [:1]8089->[:1]45926: read tcp6 [:1]8089->[:1]45926: read: connection reset
so there's maybe an entre in the 
/etc/hosts
file of your machine (from wehere you run the 
pyflyte register
command) that may look something like
Copy code
::1 localhost
If you comment it out, does it work?
Open in Slack
PreviousNext
Powered by