https://flyte.org logo
#ask-the-community
Title
# ask-the-community
b

Byron Murillo

11/18/2022, 8:18 PM
Hello guys, in the last step of the AWS manual guide after apply the changes on the config.yaml using flytectl to add the dns, I have this error on the ingresses
s

Samhita Alla

11/19/2022, 10:50 AM
@Alireza, do you any idea about why this error might be cropping up?
a

Alireza

11/19/2022, 9:42 PM
Some potential causes are: • lack of correct nodeport setting. NodePort service type is the minimum need for instance mode. The worker node instances on which the NodePort service type is installed will have a port opened so that the ALB can direct traffic there. ◦ https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/ingress/spec/https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/ingress/annotations/ • helm chart issue: Because you don't have much control over the services with the majority of pre-built helm charts. • During the building of the stack, to pass a value that is not present in the AWS Region or Account. • Before updating the resource during the stack update, erase this value from the resource's property. • in a child stack as a parameter. When a child stack's value—which is supplied from the parent stack—does not match the type of the parameter, an error results. When the resource for the parameter is missing from the account in that Region, the problem also manifests. This is really all I knew/could find, Probably @Prafulla Mahindrakar and @Sean Lin have more deep context here.
b

Byron Murillo

11/21/2022, 3:04 PM
I was reviewing all, the ALB have the correct routes, because on the pod the health checks are pinging, but any request to the dns of the alb with the /console through the browser even appears in the pod logs. An important point is in the ingress helm charts I have to put this in the annotations
<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: 'ip
, because without this the ALB is not deployed by the helm chart.
e

Eduardo Apolinario (eapolinario)

11/21/2022, 9:33 PM
cc: @Yuvraj
b

Byron Murillo

11/21/2022, 11:17 PM
I reviewed the material and is always the same, I make the manual steps on the console without using terraform putting all the tags and names correctly and get the same error, only the request from the health check gets the ingresses, idk if its for this annotation, bc without this one the alb is never deployed`alb.ingress.kubernetes.io/target-type: 'ip`
p

Prafulla Mahindrakar

11/22/2022, 2:53 PM
I think things worked before this change went in https://github.com/flyteorg/flyte/pull/2566/files And seems it wasn’t 100 % verified and was suggested this workaround https://github.com/flyteorg/flyte/issues/2743 We use internally the following ingress controller without any changes from ClusterIP to other types https://github.com/kubernetes/ingress-nginx Flyte automated setup also uses the same controller ( though this type of installation is being deprecated) https://docs.flyte.org/en/v1.0.0/deployment/aws/opta.html You can try that out
b

Byron Murillo

11/22/2022, 6:28 PM
@Prafulla Mahindrakar if I use the nginx ingress, I have to change the values in the chart for a nlb configuration right?
p

Prafulla Mahindrakar

11/22/2022, 6:49 PM
The ingress class would need to change
<http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: "nginx"
b

Byron Murillo

11/22/2022, 9:52 PM
without annotations? bc I don't found any of them on the guides
p

Prafulla Mahindrakar

11/23/2022, 6:10 AM
I don’t think we have a guide to use this particular ingress controller but would be great if you could help with a contribution to the guide once you have things working. These are sample set of annotations from gcp values which i believe should work here too for non auth enabled https://github.com/flyteorg/flyte/blob/master/charts/flyte-core/values-gcp.yaml#L150-L158 Internally we have optimized these in our deployments so you can refer to these which also has auth enabled annotations. Initially if you dont have auth enabled then you can remove the auth related annotations at the bottom
Copy code
<http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: "nginx"
        <http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
        <http://nginx.ingress.kubernetes.io/proxy-body-size|nginx.ingress.kubernetes.io/proxy-body-size>: 6m
        <http://nginx.ingress.kubernetes.io/proxy-buffer-size|nginx.ingress.kubernetes.io/proxy-buffer-size>: 16k
        <http://nginx.ingress.kubernetes.io/force-ssl-redirect|nginx.ingress.kubernetes.io/force-ssl-redirect>: "true"
        <http://nginx.ingress.kubernetes.io/proxy-cookie-domain|nginx.ingress.kubernetes.io/proxy-cookie-domain>: ~^ .$host
        <http://nginx.ingress.kubernetes.io/server-snippet|nginx.ingress.kubernetes.io/server-snippet>: |
          client_header_timeout 604800;
          client_body_timeout 604800;
          grpc_connect_timeout 1200s;
          grpc_read_timeout 1200s;
          grpc_send_timeout 1200s;
        <http://nginx.ingress.kubernetes.io/auth-url|nginx.ingress.kubernetes.io/auth-url>: "https://$host/me"
        <http://nginx.ingress.kubernetes.io/auth-signin|nginx.ingress.kubernetes.io/auth-signin>: "https://$host/login?redirect_url=$escaped_request_uri"
        <http://nginx.ingress.kubernetes.io/auth-cache-key|nginx.ingress.kubernetes.io/auth-cache-key>: $http_flyte_authorization$http_cookie
Also on side note for the aws load balancer controller seems adding this annotation mentioned here should have worked
Copy code
<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: 'ip'
https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/1695#issuecomment-850411558 You would need this on the ClusterIP services of flyte. If its worth we can debug the aws load balancer controller.
b

Byron Murillo

11/23/2022, 2:22 PM
Yesterday the ingress works only with the type anotation on the ingress, but a lot of pods has errors and cannot comunicate with others, also I noticed only the flyte-core ingress has deployed, this is because I comment all the alb configuration on the helm chart. I'm going to try the new things you send me, thanks
76 Views