Hi All, we are deploying flyte to our kubernetes cluster. We are at the SSL Certificate step. We don’t want to expose flyte to outside of our organization. But, it looks like the SSL Certificate for GKE is only for external HTTPS. Are there other options we can use so that we don’t expose flyte to outside world? Appreciate any helps from the community.

hi @Shihgian Lee let me think

@Ketan (kumare3) Thanks so much for your reply! I am not familiar with LetsEncrypt. I am working with our devops. We can look into it and then get back to you?

yes

Hi @Ketan (kumare3), thank you for taking the time to chat with us. Here is the recap of the flyte production on GKE questions we have for your team: 1. What configuration file should we be looking for to change ingress from

external

to

internal

? 2. After we have successfully changed the ingress to

internal

, we can’t use the Google Managed Cert anymore. What cert option do we have for internal ingress? Thank you in advance for your help!

cc @Prafulla Mahindrakar / @Pradithya Aria Pura/ @jeev / @Yuvraj does any of you know the answer to this. They want to enable FlyteAdmin to be within their VPN

we use an external load balancer with a google cert, but put everything behind IAP so that our users have to authenticate with google before being granted permissions to view the sites. this causes issues with talking to flyteadmin programmatically from outside our cluster. we just port-forward for that. our default dev envs which also run on GKE don't have this problem. i don't have experience running behind a VPN on GKE unfortunately, sorry.

We haven’t deployed such a setup but these instructions might help you @Shihgian Lee https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balance-ingress And adding the gce-internal annotations for the ingress https://raw.githubusercontent.com/flyteorg/flyte/master/charts/flyte-core/values-gcp.yaml and

<http://cloud.google.com/neg|cloud.google.com/neg>: '{"ingress": true}'

on the service account for flyteadmin

Thanks for the resources, @Prafulla Mahindrakar I added the annotation to `flyteadmin -> service`:

service:
    annotations:
      <http://cloud.google.com/neg|cloud.google.com/neg>: '{"ingress": true}'
      # Required for the ingress to properly route grpc traffic to grpc port
      <http://cloud.google.com/app-protocols|cloud.google.com/app-protocols>: '{"grpc":"HTTP2"}'

I did try to add it to

serviceAccount

annotation. But, it didn’t seem right based on the documentation. So, I added it to

service

instead. Under the

common->ingress

, I added and commented out a few entries based on the documentation you shared:

common:
  ingress:
    host: "{{ .Values.userSettings.hostName }}"
    tls:
      enabled: true
    annotations:
      # <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
      # <http://nginx.ingress.kubernetes.io/ssl-redirect|nginx.ingress.kubernetes.io/ssl-redirect>: "true"
      # <http://cert-manager.io/issuer|cert-manager.io/issuer>: "letsencrypt-production"
      <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: "gce-internal"
      <http://kubernetes.io/ingress.allow-http|kubernetes.io/ingress.allow-http>: "false"
      <http://kubernetes.io/ingress.global-static-ip-name|kubernetes.io/ingress.global-static-ip-name>: dev-porch-com-example
    # --- separateGrpcIngress puts GRPC routes into a separate ingress if true. Required for certain ingress controllers like nginx.
    # separateGrpcIngress: true
    # --- Extra Ingress annotations applied only to the GRPC ingress. Only makes sense if `separateGrpcIngress` is enabled.
    # separateGrpcIngressAnnotations:
      # <http://nginx.ingress.kubernetes.io/backend-protocol|nginx.ingress.kubernetes.io/backend-protocol>: "GRPC"

I managed to execute helm successfully. However, my ingress looks like the following:

kubectl -n flyte get ingress
NAME         CLASS    HOSTS                ADDRESS   PORTS     AGE
flyte-core   <none>   <http://flyte.porchpoc.com|flyte.porchpoc.com>             80, 443   45s

The

<https://flyte.porchpoc.com/console>

cannot be reached. I think it is because the

Address

is missing. There seems to be some errors for the ingress:

kubectl get events --all-namespaces --field-selector involvedObject.kind=Ingress
NAMESPACE   LAST SEEN   TYPE      REASON   OBJECT               MESSAGE
flyte       44m         Normal    Sync     ingress/flyte-core   Scheduled for sync
flyte       43m         Warning   Sync     ingress/flyte-core   Error syncing to GCP: error running load balancer syncing routine: error initializing translator env: secrets "flyte-flyte-tls" not found
flyte       5m4s        Normal    Sync     ingress/flyte-core   Scheduled for sync
flyte       2m8s        Warning   Sync     ingress/flyte-core   Error syncing to GCP: error running load balancer syncing routine: error initializing translator env: secrets "flyte-flyte-tls" not found

Any ideas what I can try next? Thanks!

Hi @Shihgian Lee, can you try turning off the tls secret in your values file https://github.com/flyteorg/flyte/blob/b28acd09b407acc2ac99535c0a16a40a051585a1/charts/flyte-core/values-gcp.yaml#L149 We have it off in other deployments too