https://flyte.org logo
Join Slack
Powered by
Hello everyone, I recently joined this channel an...
# flyte-support
g
Hello everyone, I recently joined this channel and I'm interested in working with Flyte. I’ve already deployed Flyte on EKS, using NGINX for networking. I was able to access the console in the browser successfully, and everything looks good so far! Now I moved to flyectl, there I created config file like the one below, and then tried to execute the next command:
Copy code
flytectl get projects
And then I got the next error: {"json":{},"level":"error","msg":"Connection Info: [Endpoint: flyte-flyte.test.com:443, InsecureConnection?: false, AuthMode: ClientSecret]: rpc error: code = Unknown desc = unexpected HTTP status code received from server: 464 (); malformed header: missing HTTP content-type","ts":"2025-04-15T121551+03:00"} config: 
admin:
endpoint: <http://flyte-flyte.test.com:443|flyte-flyte.test.com:443>
insecure: false
Any idea what am I missing? Thx!
g
My guess Flyte connects to GRPC endpoint. It picks up the port fine so you need not specify. Please try this admin: endpoint: dns:///flyte-flyte.test.com insecure: false
f
Good point you need to use <dns///|dns///> and no port
g
Why GRPC? In the values.yaml we have two options, GRPC or HTTP Anyway I tried with dns:/// but with no luck. This is the error I'm getting when I changed it to dns and insecure: true 
error reading server preface: http2: frame too large
And when using insecure false I see the same error 
unexpected HTTP status code received from server: 464 (); malformed header: missing HTTP content-type
This is the ingress section I use:
Copy code
ingress: {
        enabled: true,
        annotations: {
            '<http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>': '/console',
            '<http://nginx.ingress.kubernetes.io/service-upstream|nginx.ingress.kubernetes.io/service-upstream>': 'true',
            '<http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>': 'nginx-private-http',
        },
        host: interpolate`flyte-${envId}.${publicDomain()}`,
}
Maybe I missed something?
@freezing-airport-6809 appreciate your help 🙏
f
Flyte uses grpc and not http sadly
s
@gray-barista-66205 communicating with Flyte deployment is a 2 prong process. 1. HTTP coms which is used to communicate with Flyte Console. Setting this up in kubernetes is pretty straight-forward with an ingress 2. GRPC: This is not optional and is what allows flytectl and pyflyte to communicate with flyte deployment. Setting this up with a kubernetes ingress is doable, but you need to specify an ingress annotation to make sure your ingress forwards traffic using GRPC to the flyte deployment in the backend. i.e. you need to add an annotation to your ingress which states that the backend-protocol is GRPC, not HTTP. If you're using NGINX, this means adding the following annotation:
Copy code
nginx.ingress.kubernetes.io/backend-protocol: GRPC
Also, make sure that the GRPC ingress points to the port of the flyte GRPC service (usually 81 or 8089, depending on how you set things up)
g
Thanks @shy-morning-17240, I updated the grpc, but still it's not working I can see two ingress, one for http and another for grpc What am I missing?
image.png
s
What's the error you're getting?
I took out the app-root annotation from my grpc ingress (can't remember why though). I also remember having a client-side GRPC related error where GRPC tried to use it's own DNS service to find the URL and failed to find it because it needs to use the system configured DNS server not whatever python's GRPC package uses. If your error shows that your flyte client (e.g. pyflyte) is struggling to find the configured flyte-core GRPC ingress/URL, you could solve this by setting 
export GRPC_DNS_RESOLVER=native
in your terminal
If this solves your issue, then just make sure everyone adds this statement to their .bash_profile file, so that the environment variable gets set everytime a terminal is opened on a client system
this forces the grpc python package use the system-configured DNS proxy\
g
I'm trying to make sure the connection is working fine. I created a pod, and from there by using the svc I could running the next command:
Copy code
grpcurl -v -plaintext svc  list
However by using the grpc I couldn't. I assume it's something with the nginx. I will talk with the DevOps team, maybe they can help me. About removing the app-root annotation, this one needed for the http, without it you will not be able to open the flyte.
6 Views
Open in Slack
PreviousNext
Powered by