Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

I've switched to nginx-ingress, trying to get that to work now :slightly_smiling_face:.  Above question can be disregarded.

Hi Garret, I'm troubleshooting nginx-ingress. Which grpcAnnotations do you use for nginx-ingress?

I added the following yaml to my local-values.yaml and redeployed (I use flyte-binary):
ingress:
 create: true
 commonAnnotations:
  <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
 httpAnnotations:
  <http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
 grpcAnnotations:
  <http://nginx.ingress.kubernetes.io/backend-protocol|nginx.ingress.kubernetes.io/backend-protocol>: GRPC
 host: &lt;your fqdn&gt;

Then I used helm to deploy the ingress-nginx module:

# helm repo add ingress-*nginx* <https://kubernetes.github.io/ingress->*nginx*
# helm install ingress-*nginx* ingress-*nginx*/ingress-*nginx*  --version 4.2.5  --create-namespace  --namespace kube-system  --set controller.extraArgs.default-ssl-certificate="flyte/custom-default-cert"

Worked like a charm, I shouldn't have battled traffiq so long

Thank you, Garret. That's the same config which I use. I've recently removed some buffer size settings, but this seems to have no impact. So you can successfully run
grpcurl -vv yourfqdn:443 list

Hrm, that command gives me:
`Failed to list services: rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: PROTOCOL_ERROR`

You can see it is running, and all my pyflyte build and run commands work, so I'm not sure how it all fits together, other than that it appears to work.

`[ ~]# kubectl --namespace kube-system get services`
`NAME                 TYPE      CLUSTER-IP   EXTERNAL-IP  PORT(S)           AGE`
`kube-dns               ClusterIP   10.43.0.10   <none>    53/UDP,53/TCP,9153/TCP    74d`
`metrics-server            ClusterIP   10.43.79.126  <none>    443/TCP           74d`
`ingress-nginx-controller-admission  ClusterIP   10.43.168.132  <none>    443/TCP           23d`
`ingress-nginx-controller       LoadBalancer  10.43.124.77  <myip here>  80:31700/TCP,443:31796/TCP  23d`
`[ ~]# kubectl --namespace flyte get services`
`NAME          TYPE    CLUSTER-IP   EXTERNAL-IP  PORT(S)  AGE`
`flyte-binary-webhook  ClusterIP  10.43.88.15   <none>    443/TCP  27d`
`flyte-binary-grpc   ClusterIP  10.43.102.38  <none>    8089/TCP  27d`
`flyte-binary-http   ClusterIP  10.43.193.158  <none>    8088/TCP  27d`

Now it's working either with my setup, although grpcurl still fails. I definitely need to set `insecureSkipVerify: true`
Thank you, I guess settings the extraArgs option fixed the issue.