I've switched to nginx-ingress, trying to get that to work now 🙂. Above question can be disregarded.

Hi Garret, I'm troubleshooting nginx-ingress. Which grpcAnnotations do you use for nginx-ingress?

I added the following yaml to my local-values.yaml and redeployed (I use flyte-binary): ingress: create: true commonAnnotations: kubernetes.io/ingress.class: nginx httpAnnotations: nginx.ingress.kubernetes.io/app-root: /console grpcAnnotations: nginx.ingress.kubernetes.io/backend-protocol: GRPC host: <your fqdn> Then I used helm to deploy the ingress-nginx module: # helm repo add ingress-nginx https://kubernetes.github.io/ingress-*nginx* # helm install ingress-nginx ingress-*nginx*/ingress-nginx --version 4.2.5 --create-namespace --namespace kube-system --set controller.extraArgs.default-ssl-certificate="flyte/custom-default-cert" Worked like a charm, I shouldn't have battled traffiq so long

Thank you, Garret. That's the same config which I use. I've recently removed some buffer size settings, but this seems to have no impact. So you can successfully run grpcurl -vv yourfqdn:443 list

Hrm, that command gives me:

Failed to list services: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: PROTOCOL_ERROR

You can see it is running, and all my pyflyte build and run commands work, so I'm not sure how it all fits together, other than that it appears to work.

[ ~]# kubectl --namespace kube-system get services

NAME                 TYPE      CLUSTER-IP   EXTERNAL-IP  PORT(S)           AGE

kube-dns               ClusterIP   10.43.0.10   <none>    53/UDP,53/TCP,9153/TCP    74d

metrics-server            ClusterIP   10.43.79.126  <none>    443/TCP           74d

ingress-nginx-controller-admission  ClusterIP   10.43.168.132  <none>    443/TCP           23d

ingress-nginx-controller       LoadBalancer  10.43.124.77  <myip here>  80:31700/TCP,443:31796/TCP  23d

[ ~]# kubectl --namespace flyte get services

NAME          TYPE    CLUSTER-IP   EXTERNAL-IP  PORT(S)  AGE

flyte-binary-webhook  ClusterIP  10.43.88.15   <none>    443/TCP  27d

flyte-binary-grpc   ClusterIP  10.43.102.38  <none>    8089/TCP  27d

flyte-binary-http   ClusterIP  10.43.193.158  <none>    8088/TCP  27d

Now it's working either with my setup, although grpcurl still fails. I definitely need to set

insecureSkipVerify: true

Thank you, I guess settings the extraArgs option fixed the issue.