https://flyte.org logo
#ask-the-community
Title
# ask-the-community
p

Péter Kun

03/01/2024, 9:57 AM
Hi everyone, I wanted to test something basic and I wanted to leverage the local demo Flyte cluster for this: https://docs.flyte.org/en/latest/user_guide/environment_setup.html But for me even the toy example doesn't work with the demo cluster. I have started the demo cluster, exported FLYTECTL_CONFIG and got the following error:
Copy code
pyflyte run --remote hello_world.py hello_world_wf
Running Execution on Remote.
Failed with Exception Code: SYSTEM:Unknown
RPC Failed, with Status: StatusCode.UNIMPLEMENTED
        details: 
        Debug string UNKNOWN:Error received from peer  {grpc_message:"", grpc_status:12, created_time:"2024-03-01T10:46:26.305551071+01:00"}
Can you suggest something to make it work?
s

Samhita Alla

03/01/2024, 10:15 AM
could you share the config in
~/.flyte/config-sandbox.yaml
?
p

Péter Kun

03/01/2024, 10:19 AM
sure admin: # For GRPC endpoints you might want to use dns:///flyte.myexample.com endpoint: localhost:30080 insecure: true # This is not a needed configuration, only useful if you want to explore the data in sandbox. For non sandbox, please # do not use this configuration, instead prefer to use aws, gcs, azure sessions. Flytekit, should use fsspec to # auto select the right backend to pull data as long as the sessions are configured. For Sandbox, this is special, as # minio is s3 compatible and we ship with minio in sandbox. storage: connection: endpoint: http://localhost:30002 access-key: minio secret-key: miniostorage
s

Samhita Alla

03/01/2024, 11:01 AM
have you tried tearing down your cluster and restarting it?
flytectl demo teardown
and
flytectl demo start
p

Péter Kun

03/01/2024, 11:06 AM
I just did it and I am facing the same error
s

Samhita Alla

03/01/2024, 12:32 PM
what's the output of
kubectl get pods -n flyte
?
are all the pods running?
p

Péter Kun

03/01/2024, 1:25 PM
yes,
Copy code
NAME                                                  READY   STATUS    RESTARTS       AGE
flyte-sandbox-docker-registry-57789f56d-27zcw         1/1     Running   0              141m
flyte-sandbox-kubernetes-dashboard-6757db879c-nkmgs   1/1     Running   0              141m
flyte-sandbox-postgresql-0                            1/1     Running   0              141m
flyte-sandbox-buildkit-675bff7bd9-l58vv               1/1     Running   0              141m
flyte-sandbox-minio-85dc5446b6-lw9f7                  1/1     Running   0              141m
flyteagent-b98cfb65f-2dxjb                            1/1     Running   0              141m
flyte-sandbox-proxy-8f4f7fd9b-hpvl4                   1/1     Running   0              141m
flyte-sandbox-7f89d97699-67bwc                        1/1     Running   2 (140m ago)   141m
s

Samhita Alla

03/02/2024, 4:56 PM
@David Espejo (he/him) i'm running out of ways to debug this. any idea what might be causing this issue?
d

David Espejo (he/him)

03/04/2024, 4:47 PM
@Péter Kun could you please install
grpcurl
to validate any possible connection error to the gRPC server:
Copy code
grpcurl --plaintext -v localhost:30080 list
It should output
Copy code
flyteidl.service.AdminService
flyteidl.service.DataProxyService
flyteidl.service.SignalService
grpc.health.v1.Health
grpc.reflection.v1.ServerReflection
grpc.reflection.v1alpha.ServerReflection
also see if
localhost
is actually pointing to
127.0.0.1
in your system and that the `30080`port is not used
p

Péter Kun

03/06/2024, 3:57 PM
You are right, I think the problem was that `30080`port was occupied. Is there some way to customize this port (to use 30081 port instead) in the demo cluster?
d

David Espejo (he/him)

03/06/2024, 4:23 PM
I think there were discussions about that in the past but no straightforward mechanism to do so. I tried invoking a config file as part of the
flytectl demo start -c <config>
command but it gets rewritten
p

Péter Kun

03/07/2024, 11:45 AM
I see, thank you for your help!