Hi First time trying to deploy Flyte on AWS following these

Question

Hi First time trying to deploy Flyte on AWS following these instructions After installing the Helm chart and verifying that the pods are running I fail to get an address for my hosts ```$ kubectl get ingress n flyte NAME CLASS HOSTS ADDRESS PORTS AGE flyte core lt none gt 80 26m flyte core grpc lt none gt 80 26m``` Describing the ingress shows this ```kubectl describe ingress n flyte Name flyte core Namespace flyte Address Default backend default http backend 80 lt error endpoints default http backend not found gt Rules Host Path Backends ssl redirect use annotation lt error endpoints ssl redirect not found gt console flyteconsole 80 192 168 103 57 8080 192 168 128 109 8080 console flyteconsole 80 192 168 103 57 8080 192 168 128 109 8080 api flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 api flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 healthcheck flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 v1 flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 well known flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 well known flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 login flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 login flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 logout flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 logout flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 callback flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 callback flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 me flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 config flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 config flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 oauth2 flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 oauth2 flyteadmin 80 192 168 110 232 8088 192 168 152 242 8088 Annotations Type redirect RedirectConfig Protocol HTTPS Port 443 StatusCode HTTP 301 arn aws acm us west 2 582526512915 certificate 6c75c8f4 04a1 4aa7 81fa 59c7241e52ba flyte HTTP 80 HTTPS 443 internet facing service instance=production alb flyte flyte console Events Type Reason Age From Message Warning FailedDeployModel 5m1s x18 over 26m ingress Failed deploy model due to InvalidParameter 1 validation error s found minimum field value of 1 CreateTargetGroupInput Port Name flyte core grpc Namespace flyte Address Default backend default http backend 80 lt error endpoints default http backend not found gt Rules Host Path Backends flyteidl service AdminService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 flyteidl service AdminService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 flyteidl service DataProxyService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 flyteidl service DataProxyService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 flyteidl service AuthMetadataService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 flyteidl service AuthMetadataService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 flyteidl service IdentityService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 flyteidl service IdentityService flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 grpc health v1 Health flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 grpc health v1 Health flyteadmin 81 192 168 110 232 8089 192 168 152 242 8089 Annotations Type redirect RedirectConfig Protocol HTTPS Port 443 StatusCode HTTP 301 GRPC arn aws acm us west 2 582526512915 certificate 6c75c8f4 04a1 4aa7 81fa 59c7241e52ba flyte HTTP 80 HTTPS 443 internet facing service instance=production alb flyte flyte console GRPC Events Type Reason Age From Message Warning FailedDeployModel 5m4s x19 over 27m ingress Failed deploy model due to InvalidParameter 1 validation error s found minimum field value of 1 CreateTargetGroupInput Port ``` Any ideas how to best debug

Yee · Answer

those instructions are a bit dated unf we ll look to clean them up soon hopefully

Yee · Answer

they shouldn t have annotations for both nginx and alb

Yee · Answer

what ingress controller are you using alb i assume since that s what the instructions have

Lawrence Lee · Answer

yes using ALB

Lawrence Lee · Answer

is there anything I can change in `values eks yaml` as a workaround

Yee · Answer

sorry was afk

Yee · Answer

looking a bit more now at their docs

Yee · Answer

are there any logs from the ingress controller itself

Lawrence Lee · Answer

how can I access those logs I m a bit of a k8s noob

Lawrence Lee · Answer

sorry that was an easy thing to do sweat smile

Lawrence Lee · Answer

``` level error ts 1671213485 8348856 logger controller ingress msg Reconciler error name flyte namespace error InvalidParameter 1 validation error s found n minimum field value of 1 CreateTargetGroupInput Port n ```

Lawrence Lee · Answer

this was the only error log entry

Yee · Answer

can you get o yaml the ingress also

Yee · Answer

both of them flyte core and the grpc one

Yee · Answer

<https nubisoft io blog how to set up kubernetes ingress with aws alb ingress controller |this here> might be relevant

Yee · Answer

subnet needs to be tagged with `<http kubernetes io role elb=1|kubernetes io role elb=1>`

Yee · Answer

sorry we ve been using nginx as the ingress controller for a spell now been a while since i ve played around with alb

Yee · Answer

we honestly need to delete our instructions and just point users to the <https aws amazon com blogs opensource kubernetes ingress aws alb ingress controller |official docs> esp since we have no way of knowing or keeping up with the updates

Yee · Answer

can you also `k n flyte get service o wide`

Lawrence Lee · Answer

do you mean pass ` o yaml` to the `k describe ingress` call

Lawrence Lee · Answer

```kubectl get service n flyte o wide NAME TYPE CLUSTER IP EXTERNAL IP PORT S AGE SELECTOR datacatalog NodePort 10 100 184 184 lt none gt 8089 32690 TCP 88 31839 TCP 89 32201 TCP 3h12m flyte pod webhook ClusterIP 10 100 226 35 lt none gt 443 TCP 3h12m app=flyte pod webhook flyteadmin ClusterIP 10 100 200 84 lt none gt 80 TCP 81 TCP 87 TCP 10254 TCP 3h12m flyteconsole ClusterIP 10 100 211 80 lt none gt 80 TCP 3h12m ```

Lawrence Lee · Answer

thanks so much for the help pray

Yee · Answer

k n flyte get ingress o yaml

Yee · Answer

yeah i don t think the datacatalog service should be a nodeport

Yee · Answer

Lawrence Lee · Answer

```apiVersion v1 items apiVersion kind Ingress metadata annotations Type redirect RedirectConfig Protocol HTTPS Port 443 StatusCode HTTP 301 arn aws acm us west 2 certificate 6c75c8f4 04a1 4aa7 81fa 59c7241e52ba flyte HTTP 80 HTTPS 443 internet facing service instance=production alb flyte flyte console creationTimestamp 2022 12 16T17 58 03Z finalizers group ingress k8s aws flyte generation 1 labels Helm name flyte core namespace flyte resourceVersion 408320 uid 3d84d59a 08fe 41d9 bea3 305413ddd483 spec rules http paths backend service name ssl redirect port name use annotation path pathType ImplementationSpecific backend service name flyteconsole port number 80 path console pathType ImplementationSpecific backend service name flyteconsole port number 80 path console pathType ImplementationSpecific backend service name flyteadmin port number 80 path api pathType ImplementationSpecific backend service name flyteadmin port number 80 path api pathType ImplementationSpecific backend service name flyteadmin port number 80 path healthcheck pathType ImplementationSpecific backend service name flyteadmin port number 80 path v1 pathType ImplementationSpecific backend service name flyteadmin port number 80 path well known pathType ImplementationSpecific backend service name flyteadmin port number 80 path well known pathType ImplementationSpecific backend service name flyteadmin port number 80 path login pathType ImplementationSpecific backend service name flyteadmin port number 80 path login pathType ImplementationSpecific backend service name flyteadmin port number 80 path logout pathType ImplementationSpecific backend service name flyteadmin port number 80 path logout pathType ImplementationSpecific backend service name flyteadmin port number 80 path callback pathType ImplementationSpecific backend service name flyteadmin port number 80 path callback pathType ImplementationSpecific backend service name flyteadmin port number 80 path me pathType ImplementationSpecific backend service name flyteadmin port number 80 path config pathType ImplementationSpecific backend service name flyteadmin port number 80 path config pathType ImplementationSpecific backend service name flyteadmin port number 80 path oauth2 pathType ImplementationSpecific backend service name flyteadmin port number 80 path oauth2 pathType ImplementationSpecific status loadBalancer apiVersion kind Ingress metadata annotations Type redirect RedirectConfig Protocol HTTPS Port 443 StatusCode HTTP 301 GRPC arn aws acm us west 2 certificate 6c75c8f4 04a1 4aa7 81fa 59c7241e52ba flyte HTTP 80 HTTPS 443 internet facing service instance=production alb flyte flyte console GRPC creationTimestamp 2022 12 16T17 58 03Z finalizers group ingress k8s aws flyte generation 1 labels Helm name flyte core grpc namespace flyte resourceVersion 408218 uid 114b0b7e ede6 43f1 8baa 7871600b81c5 spec rules http paths backend service name flyteadmin port number 81 path flyteidl service AdminService pathType ImplementationSpecific backend service name flyteadmin port number 81 path flyteidl service AdminService pathType ImplementationSpecific backend service name flyteadmin port number 81 path flyteidl service DataProxyService pathType ImplementationSpecific backend service name flyteadmin port number 81 path flyteidl service DataProxyService pathType ImplementationSpecific backend service name flyteadmin port number 81 path flyteidl service AuthMetadataService pathType ImplementationSpecific backend service name flyteadmin port number 81 path flyteidl service AuthMetadataService pathType ImplementationSpecific backend service name flyteadmin port number 81 path flyteidl service IdentityService pathType ImplementationSpecific backend service name flyteadmin port number 81 path flyteidl service IdentityService pathType ImplementationSpecific backend service name flyteadmin port number 81 path grpc health v1 Health pathType ImplementationSpecific backend service name flyteadmin port number 81 path grpc health v1 Health pathType ImplementationSpecific status loadBalancer kind List metadata resourceVersion selfLink ```

Lawrence Lee · Answer

curious has anyone attempted an ECS deployment of Flyte Could be a good option for AWS folks who are not as comfortable with k8s and potentially much more amenable to Cloudformation Terraform style deployment

Lawrence Lee · Answer

Ok confirmed that the Github issue fix did the trick for me

Lawrence Lee · Answer

adding ``` <http alb ingress kubernetes io target type|alb ingress kubernetes io target type> ip``` to the annotations created the ALB correctly

Yee · Answer

sorry working on another issue right now let me come back to this in a bit

Lawrence Lee · Answer

I did not have to tag my subnets FWIW

Yee · Answer

oh nice

Yee · Answer

good to know

Lawrence Lee · Answer

now to work on authentication

Lawrence Lee · Answer

thanks < Yee> and hope this thread is useful for anyone encountering the same problem in the future

Ketan (kumare3) · Answer

Great to see you in the channel < Lawrence Lee> thank you for trying out Flyte flyte

Ferdinand von den Eichen · Answer

I followed the same deployment guide today and had to make the same adjustment i e adding ``` ip``` to the `eks values yaml` That is really lovely and much more helpful than the on the latest website I also found that the guide was apart from that flag above in great shape still Any thoughts on adding it to the current version of the website

Ketan (kumare3) · Answer

thank you for the comment < Ferdinand von den Eichen> the old guide we felt was overly verbose for most people

Ketan (kumare3) · Answer

that is why we made a simpler one

David Espejo (he/him) · Answer

cc < Niels Bantilan>

Niels Bantilan · Answer

gt the old guide we felt was overly verbose for most people yeah on the other hand it sounds like it helped people get setup based on < Ferdinand von den Eichen> s feedback it may be worth adding at least a link to those manual setup docs as a stop gap For context Ferdinand maintaining those guides manually was very challenging however if we do put up a link to those legacy guides would you be able to help us figure out what s out dated We are working on a for AWS and GCP via terraform which will nullify the need for the manual docs setup and will simplify how you can stand up a Flyte cluster on the cloud

Ferdinand von den Eichen · Answer

I totally understand the effort scream at the same time the high level current guide was good for getting a basic understanding of all pieces involved but lacks many of the crucial details to ensure a smooth first setup I could imagine that Flyte is about to gain a lot of traction so first setup experience should be smooth sailing on the major clouds The Issue you linked sounds incredibly promising in that regard though so I ll leave it up to you guys if you think it will cover the new user path sufficiently It sounds like it does actually Regarding the outdated article The only change I needed to make was adding ```<http alb ingress kubernetes io target type|alb ingress kubernetes io target type> ip``` to the `eks values yaml` Everything else was still on point

Ketan (kumare3) · Answer

< Ferdinand von den Eichen> honestly if you can help maintain we can keep the article as under the hood

Ferdinand von den Eichen · Answer

I we at Kineo don t mind contributing back at all that s the magic of OS right heart hands That being said I m not sure how easy it is to contribute here since it is more than just a PR against README md Can you point me in the right direction

Ketan (kumare3) · Answer

Cc < David Espejo he him > please comment

Ketan (kumare3) · Answer

But we are a very open community and love contributions

David Espejo (he/him) · Answer

We do < Ferdinand von den Eichen> Here some instructions to contribute to docs If any question let us know

Ferdinand von den Eichen · Answer

I started a PR here <https github com flyteorg flyte pull 3343> After checking the docs again I realised that the change is actually not needed in the docs but rather in the eks values yaml itself I opened a PR any help is appreciated In particular with making sure relevant tests run if there are any for the EKS deployment

Ketan (kumare3) · Answer

< David Espejo he him > can you help merge this please

Alex Papanicolaou · Answer

< Ferdinand von den Eichen> I m a bit confused This said to add that annotation to the service so putting it under `flyteadmin service annotations` and `flyteconsole service annotations` But apparently it ll just work as an annotation on the ingress

David Espejo (he/him) · Answer

< Alex Papanicolaou> the `` annotation can be applied both at the Service and Ingress level with a higher precedence for the Service level In this case the annotation for Ingress will be applied unless there s another at the Service level but I guess due to the Flyte architecture and how Ingress is the main entrance point from flytekit it makes sense to just annotate the Ingress resource

Alex Papanicolaou · Answer

Thanks < David Espejo he him > That makes sense and makes the ingress setup simple