https://flyte.org logo
Join Slack
Powered by
Hi again, Not sure if this is directly related to ...
# flyte-deployment
g
Hi again, Not sure if this is directly related to flyte.. I have installed a brand new EKS cluster, with the aws-load-balancer-controller installed for creating ALB's for flyte's Ingress. However, the controller is not able to provision an ALB, because it creates an invalid configuration for it The error shown by the alb controller is:
Copy code
{
  "level": "error",
  "ts": "2023-09-03T12:17:51Z",
  "msg": "Reconciler error",
  "controller": "ingress",
  "object": {
    "name": "flyte-backend-flyte-binary-http",
    "namespace": "flyte"
  },
  "namespace": "flyte",
  "name": "flyte-backend-flyte-binary-http",
  "reconcileID": "65bc9264-18ff-4930-b29b-ac1abf7cf524",
  "error": "InvalidParameter: 1 validation error(s) found.\n- minimum field value of 1, CreateTargetGroupInput.Port.\n"
}
Probably due to this section in the configuration it is trying to implement:
Copy code
"AWS::ElasticLoadBalancingV2::TargetGroup": {
      "flyte/flyte-backend-flyte-binary-http-flyte-backend-flyte-binary-http:8088": {
        "spec": {
          "name": "k8s-flyte-flytebac-743fe1ff44",
          "targetType": "instance",
          "port": 0,
          "protocol": "HTTP",
          "protocolVersion": "HTTP1",
          "ipAddressType": "ipv4",
          "healthCheckConfig": {
            "port": "traffic-port",
            "protocol": "HTTP",
            "path": "/",
            "matcher": {
              "httpCode": "200"
            },
            "intervalSeconds": 15,
            "timeoutSeconds": 5,
            "healthyThresholdCount": 2,
            "unhealthyThresholdCount": 2
          }
        }
      }
    },
Notice "port: 0" in there, I think that's what it's choking on. But I'm not sure why it's not picking up the correct port... The current ingress for the http backend is:
Copy code
Name:             flyte-backend-flyte-binary-http
Labels:           <http://app.kubernetes.io/instance=flyte-backend|app.kubernetes.io/instance=flyte-backend>
                  <http://app.kubernetes.io/managed-by=Helm|app.kubernetes.io/managed-by=Helm>
                  <http://app.kubernetes.io/name=flyte-binary|app.kubernetes.io/name=flyte-binary>
                  <http://app.kubernetes.io/version=1.16.0|app.kubernetes.io/version=1.16.0>
                  <http://helm.sh/chart=flyte-binary-v1.9.1|helm.sh/chart=flyte-binary-v1.9.1>
Namespace:        flyte
Address:          
Ingress Class:    <none>
Default backend:  <default>
Rules:
  Host                       Path  Backends
  ----                       ----  --------
  <http://staging.flyte.brain.space|staging.flyte.brain.space>  
                             /console         flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /console/*       flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /api             flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /api/*           flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /healthcheck     flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /v1/*            flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /.well-known     flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /.well-known/*   flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /login           flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /login/*         flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /logout          flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /logout/*        flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /callback        flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /callback/*      flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /me              flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /config          flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /config/*        flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /oauth2          flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
                             /oauth2/*        flyte-backend-flyte-binary-http:8088 (10.0.60.54:8088)
Annotations:                 <http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: instance
                             <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
                             <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: flyte-backend
                             <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: flyte
                             <http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
Events:
  Type     Reason             Age                  From     Message
  ----     ------             ----                 ----     -------
  Warning  FailedDeployModel  14m (x23 over 3h6m)  ingress  Failed deploy model due to InvalidParameter: 1 validation error(s) found.
- minimum field value of 1, CreateTargetGroupInput.Port.
Any idea?...
So it turns out the mistake was missing annotations in values.yaml.... The following configuration seems to work
Copy code
ingress:
  create: true
  commonAnnotations:
    <http://alb.ingress.kubernetes.io/certificate-arn|alb.ingress.kubernetes.io/certificate-arn>: '${ssl_cert_arn}'
    <http://alb.ingress.kubernetes.io/group.name|alb.ingress.kubernetes.io/group.name>: flyte
    <http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: '[{"HTTP": 80}, {"HTTPS":443}]'
    <http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
    <http://alb.ingress.kubernetes.io/ssl-redirect|alb.ingress.kubernetes.io/ssl-redirect>: '443'
    <http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: ip
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
d
right @Guy Harel this is in process of being added to the docs
2 Views
Open in Slack
PreviousNext
Powered by