Hi, i m trying to deploy flyte-core on an AKS cluster that uses nginx-ingress-controller and to handle grpc routing I used

separateGrpcIngress: true

, so nginx controller can handle http and http2. However using same fqdn does not make nginx-ingress-controller happy really.. I m getting

Warning  Rejected  8m28s  nginx-ingress-controller  All hosts are taken by other resources

error on one of the ingress objects (depending on which one is applied first, it changes.) Apparently nginx does not support multiple ingress objects for the same host/fqdn. When I try to deploy with only one ingress object, then grpc protocol is not supported. So, I was wondering if anyone made it work with some kind of magical configuration. PS: I tried

mergeable-ingress-types

as well, but no luck :(

@hfurkanvural I still need to test this with flyte-core but take a look at these annotations: https://github.com/flyteorg/flyte/blob/243a8cb3e38b09307c40bc24b206f408a1192d9f/charts/flyte-core/values-gcp.yaml#L151-L164 (except for the one about

cert-manager

)

@David Espejo (he/him) Thanks, i tried with that config and unfortunately it s still the same 😞 I guess nginx-ingress is not an option in this case

@hfurkanvural If still relevant for you - we had the same issue we resolved it by disabling ingress in the helm values and manually defining a single ingress for flyte-http and flyte-grpc. But we are using nginx plus (annotations) which has different annotations for grpc.

apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>

kind: Ingress metadata: annotations: nginx.org/grpc-services: "flyte-grpc" name: flyte spec: ingressClassName: nginx rules: - host: flyte.my.domain.dev http: paths: - backend: service: name: flyte-http port: number: 8088 path: /console pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /console/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /api pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /api/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /healthcheck pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /v1/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /.well-known pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /.well-known/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /login pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /login/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /logout pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /logout/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /callback pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /callback/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /me pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /config pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /config/* pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /oauth2 pathType: ImplementationSpecific - backend: service: name: flyte-http port: number: 8088 path: /oauth2/* pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.AdminService pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.AdminService/* pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.DataProxyService pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.DataProxyService/* pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.AuthMetadataService pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.AuthMetadataService/* pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.IdentityService pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.IdentityService/* pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /grpc.health.v1.Health pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /grpc.health.v1.Health/* pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.SignalService pathType: ImplementationSpecific - backend: service: name: flyte-grpc port: number: 8089 path: /flyteidl.service.SignalService/* pathType: ImplementationSpecific tls: - hosts: - flyte.my.domain.dev

@Clemens thanks for the help. In the end we couldnt find a solution and used traefik with ingressroute definitions for grpc paths. But i will definitely give this a try in the future. Maybe you can also create a PR for this change?

@hfurkanvural you mean for the chart? - we are using flyte-binary chart - this case seams very specific (in case your kubernetes/ingress/cert-manager setup - can not merge/handle multiple ingress having the same tls host) - I think its fine if you disable ingress creation in the helm chart and provide the ingress differently.

Oh i see. Let me try it myself and see if i can do sth about this in the chart. Thanks again for the help ✌🏼