Hi All, I am trying to POC in the restricted env using single cluster deployment. In our environment, We cannot give permissions to create namespace and service accounts to flyte's system level service account. We can create namespace and service accounts using pipelines. Is there a way to configure flyte to use existing namespace and service accounts for flyte project. Right now it is trying to create namespaces and service accounts even if they exist?

Yes, Flyte can be restricted to one or per domain namespace (recommended)

@peeyush jain you can prevent Flyte from creating service accounts, you can also as Ketan indicated, make Flyte use a single namespace but I'm not sure if what you need is to prevent Flyte from creating any namespace by default?

@David Espejo (he/him) Thanks for response. How can i prevent flyte from creating service accounts? what config and value do i need to achieve that? Regarding the namespace i can create namespaces using pipelines. Will it possible to use pre-created namespaces?

@peeyush jain you can disable SA creation by setting the following key to `false`: https://github.com/flyteorg/flyte/blob/a2a63eb893bdff3c2d1a01242d0d25b73f287b1f/charts/flyte-binary/values.yaml#L371-L373 Or this key for flyteadmin and flytepropeller, in cas you're using the flyte-core Helm chart: https://github.com/flyteorg/flyte/blob/a2a63eb893bdff3c2d1a01242d0d25b73f287b1f/charts/flyte-core/values-eks.yaml#L19

Thanks. I am deploying on EKS in aws. I am using eks-starter.yaml for helm chart deployment. I am also looking for any guide to use AWS secret manager instead of K8s for secrets pluglin. Any pointers will be helpful.

no guide yet but a workaround proposed here: https://github.com/flyteorg/flyte/issues/4336

Thanks. For time being i got admin to open up permissions on cluster roles. I was able to complete the deployment. Now when i try to flytectl it throws error flytectl -p flytesnacks -d development get tasks Roc error: code = unavailable desc = connection error: desc = \”transport: authentication handshake failed: context deadline exceeded” Using below admin conf endpoint: dns:///ml-platform-poc.xxxxxxxx.xxx authType: pkce insecure: false insecureSkipVerify: true Pyflyte run —remote example.py hello_wrold_wf fails with Debug string UNKNOWN: Error received from peer {gprc_message:”Received http2 header with status 464, sprc_status:2 created_time”2024-03-01T112226.47676767+00:00”}

@peeyush jain the ALB config on these lines is typically enough to allow you to terminate at the load balancer using an ACM-managed certificate

Thanks @David Espejo (he/him) In our env, we are not allowed to alb as native ingress controller. We create alb and targetgroups and bindings externally via tfe. Nginx is supported as native controller but it does not support grpc protocol in our environment. With nginx flyte console works fine but when using flytectl i get error "recieved http2 header, status 464". Hence i switched to external alb for ingress which allows grpc but ssl has to terminate on pod. I am using single cluster deployment eks-starter.yaml. Will need your inputs on how to configure ssl for flyte binary pod.

I've seen situations with other Ingress controllers that switched to NGINX because it supports gRPC. Using these annotations should work:

ingressClassName: nginx
  commonAnnotations:
    ingress.kubernetes.io/rewrite-target: /
  httpAnnotations:
    nginx.ingress.kubernetes.io/app-root: /console
  grpcAnnotations:
    nginx.ingress.kubernetes.io/backend-protocol: GRPC

Thanks David. I will create a ssl proxy service and see if it works