Hello all! I’m deploying flyte to aws eks. I follo...
# flyte-deployment
e
Hello all! I’m deploying flyte to aws eks. I followed the instructions in the docs, and everything seems to be set up. However, the ALB ingress controller is not giving me an address.
Copy code
$ kubectl -n flyte get ingress
NAME              CLASS    HOSTS   ADDRESS   PORTS   AGE
flyte-core        <none>   *                 80      41m
flyte-core-grpc   <none>   *                 80      41m
This issue was mentioned in the troubleshooting section of the docs, and it suggested running
Copy code
$ kubectl describe ingress -n flyte
Name:             flyte-core
Labels:           <http://app.kubernetes.io/managed-by=Helm|app.kubernetes.io/managed-by=Helm>
Namespace:        flyte
Address:          
Ingress Class:    <none>
Default backend:  <default>
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           
              /*               ssl-redirect:use-annotation (<error: endpoints "ssl-redirect" not found>)
              
...

Annotations:  <http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>:
                {"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}
              <http://alb.ingress.kubernetes.io/certificate-arn|alb.ingress.kubernetes.io/certificate-arn>: arn:aws:acm:us-east-1:752578504353:certificate/7a7065cb-1ffc-418e-8070-bc36fbaff7cb
              <http://alb.ingress.kubernetes.io/group.name|alb.ingress.kubernetes.io/group.name>: flyte
              <http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: [{"HTTP": 80}, {"HTTPS":443}]
              <http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
              <http://alb.ingress.kubernetes.io/tags|alb.ingress.kubernetes.io/tags>: service_instance=production
              <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
              <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: flyte
              <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: flyte
              <http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
Events:
  Type     Reason             Age                   From     Message
  ----     ------             ----                  ----     -------
  Warning  FailedDeployModel  3m33s (x20 over 42m)  ingress  Failed deploy model due to InvalidParameter: 1 validation error(s) found.
- minimum field value of 1, CreateTargetGroupInput.Port.


Name:             flyte-core-grpc
Labels:           <http://app.kubernetes.io/managed-by=Helm|app.kubernetes.io/managed-by=Helm>
Namespace:        flyte
Address:          
Ingress Class:    <none>
Default backend:  <default>
Rules:
  Host        Path  Backends
  ----        ----  --------

  ...

Annotations:  <http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>:
               {"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}

...

Events:
  Type     Reason             Age                   From     Message
  ----     ------             ----                  ----     -------
  Warning  FailedDeployModel  3m36s (x19 over 42m)  ingress  Failed deploy model due to InvalidParameter: 1 validation error(s) found.
- minimum field value of 1, CreateTargetGroupInput.Port.
Could someone point me to the right direction in debugging this? I checked the security groups of EKS cluster and RDS, they were the same. Thanks in advance. Best, Ekku, CTO & Co-Founder @ inven.ai
s
Hi Ekku, thanks for sharing your issue. Can you check if the target group binding for the ALB is correctly configured once
e
Hey, thanks for the follow up 🙂 I will check this first thing tomorrow
Hey @Shivay Lamba, sorry for the delay. It seems like the ALB wasn’t created, which would explain the issue, I suppose 😄. Although I’m not sure why - I believe I followed the steps and everything seemed fine.
s
Okay could you try to check through the steps and setup the ALB? Let us know if you face any issues.
e
I tried to run through the steps again, still no address showing up on
kubectl -n flyte get ingress
. @Shivay Lamba can you specify a bit more on how would I check the target group binding?
s
Hi Ekku, let me get back to you
l
When I saw this, I had to add
<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: 'ip'
to the ingress in values-eks.yaml
e
For sure @Shivay Lamba!
@Laura Lin Thanks for the tip! I actually saw someone else had suggested that in #ask-the-community. I tried that, didn’t work immediately, though. Did you have to adjust some other configurations for this? I saw that AWS Load Balancer Controller docs say that:
Copy code
IP mode
Ingress traffic starts at the ALB and reaches the Kubernetes pods directly. CNIs must support directly accessible POD ip via secondary IP addresses on ENI.
was the ENI configured correctly for this automatically?
l
No, once I added that setting to the ingress and upgraded the helm, I was able to see it.
e
Hmm, okay. I’ll have to keep looking. Thanks!
167 Views