faint-egg-81361
12/16/2022, 6:36 PM$ kubectl get ingress -n flyte
NAME CLASS HOSTS ADDRESS PORTS AGE
flyte-core <none> * 80 26m
flyte-core-grpc <none> * 80 26m
Describing the ingress shows this
kubectl describe ingress -n flyte
Name: flyte-core
Namespace: flyte
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/* ssl-redirect:use-annotation (<error: endpoints "ssl-redirect" not found>)
/console flyteconsole:80 (192.168.103.57:8080,192.168.128.109:8080)
/console/* flyteconsole:80 (192.168.103.57:8080,192.168.128.109:8080)
/api flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/api/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/healthcheck flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/v1/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/.well-known flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/.well-known/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/login flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/login/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/logout flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/logout/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/callback flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/callback/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/me flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/config flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/config/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/oauth2 flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
/oauth2/* flyteadmin:80 (192.168.110.232:8088,192.168.152.242:8088)
Annotations: <http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>:
{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}
<http://alb.ingress.kubernetes.io/certificate-arn|alb.ingress.kubernetes.io/certificate-arn>: arn:aws:acm:us-west-2:582526512915:certificate/6c75c8f4-04a1-4aa7-81fa-59c7241e52ba
<http://alb.ingress.kubernetes.io/group.name|alb.ingress.kubernetes.io/group.name>: flyte
<http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: [{"HTTP": 80}, {"HTTPS":443}]
<http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
<http://alb.ingress.kubernetes.io/tags|alb.ingress.kubernetes.io/tags>: service_instance=production
<http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
<http://meta.helm.sh/release-name|meta.helm.sh/release-name>: flyte
<http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: flyte
<http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedDeployModel 5m1s (x18 over 26m) ingress Failed deploy model due to InvalidParameter: 1 validation error(s) found.
- minimum field value of 1, CreateTargetGroupInput.Port.
Name: flyte-core-grpc
Namespace: flyte
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/flyteidl.service.AdminService flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/flyteidl.service.AdminService/* flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/flyteidl.service.DataProxyService flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/flyteidl.service.DataProxyService/* flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/flyteidl.service.AuthMetadataService flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/flyteidl.service.AuthMetadataService/* flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/flyteidl.service.IdentityService flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/flyteidl.service.IdentityService/* flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/grpc.health.v1.Health flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
/grpc.health.v1.Health/* flyteadmin:81 (192.168.110.232:8089,192.168.152.242:8089)
Annotations: <http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>:
{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}
<http://alb.ingress.kubernetes.io/backend-protocol-version|alb.ingress.kubernetes.io/backend-protocol-version>: GRPC
<http://alb.ingress.kubernetes.io/certificate-arn|alb.ingress.kubernetes.io/certificate-arn>: arn:aws:acm:us-west-2:582526512915:certificate/6c75c8f4-04a1-4aa7-81fa-59c7241e52ba
<http://alb.ingress.kubernetes.io/group.name|alb.ingress.kubernetes.io/group.name>: flyte
<http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: [{"HTTP": 80}, {"HTTPS":443}]
<http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
<http://alb.ingress.kubernetes.io/tags|alb.ingress.kubernetes.io/tags>: service_instance=production
<http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
<http://meta.helm.sh/release-name|meta.helm.sh/release-name>: flyte
<http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: flyte
<http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
<http://nginx.ingress.kubernetes.io/backend-protocol|nginx.ingress.kubernetes.io/backend-protocol>: GRPC
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedDeployModel 5m4s (x19 over 27m) ingress Failed deploy model due to InvalidParameter: 1 validation error(s) found.
- minimum field value of 1, CreateTargetGroupInput.Port.
Any ideas how to best debug?thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
faint-egg-81361
12/16/2022, 7:23 PMfaint-egg-81361
12/16/2022, 7:44 PMvalues-eks.yaml
as a workaround?thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
faint-egg-81361
12/16/2022, 7:46 PMfaint-egg-81361
12/16/2022, 8:05 PMfaint-egg-81361
12/16/2022, 8:05 PM{
"level": "error",
"ts": 1671213485.8348856,
"logger": "controller.ingress",
"msg": "Reconciler error",
"name": "flyte",
"namespace": "",
"error": "InvalidParameter: 1 validation error(s) found.\n- minimum field value of 1, CreateTargetGroupInput.Port.\n"
}
faint-egg-81361
12/16/2022, 8:14 PMthankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
<http://kubernetes.io/role/elb=1|kubernetes.io/role/elb=1>
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
k -n flyte get service -o wide
faint-egg-81361
12/16/2022, 9:09 PM-o yaml
to the k describe ingress
call?faint-egg-81361
12/16/2022, 9:10 PMkubectl get service -n flyte -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
datacatalog NodePort 10.100.184.184 <none> 8089:32690/TCP,88:31839/TCP,89:32201/TCP 3h12m <http://app.kubernetes.io/instance=flyte,app.kubernetes.io/name=datacatalog|app.kubernetes.io/instance=flyte,app.kubernetes.io/name=datacatalog>
flyte-pod-webhook ClusterIP 10.100.226.35 <none> 443/TCP 3h12m app=flyte-pod-webhook
flyteadmin ClusterIP 10.100.200.84 <none> 80/TCP,81/TCP,87/TCP,10254/TCP 3h12m <http://app.kubernetes.io/instance=flyte,app.kubernetes.io/name=flyteadmin|app.kubernetes.io/instance=flyte,app.kubernetes.io/name=flyteadmin>
flyteconsole ClusterIP 10.100.211.80 <none> 80/TCP 3h12m <http://app.kubernetes.io/instance=flyte,app.kubernetes.io/name=flyteconsole|app.kubernetes.io/instance=flyte,app.kubernetes.io/name=flyteconsole>
faint-egg-81361
12/16/2022, 9:11 PMthankful-minister-83577
thankful-minister-83577
thankful-minister-83577
faint-egg-81361
12/16/2022, 9:15 PMapiVersion: v1
items:
- apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
annotations:
<http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>: '{"Type": "redirect", "RedirectConfig":
{ "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
<http://alb.ingress.kubernetes.io/certificate-arn|alb.ingress.kubernetes.io/certificate-arn>: arn:aws:acm:us-west-2:*************:certificate/6c75c8f4-04a1-4aa7-81fa-59c7241e52ba
<http://alb.ingress.kubernetes.io/group.name|alb.ingress.kubernetes.io/group.name>: flyte
<http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: '[{"HTTP": 80}, {"HTTPS":443}]'
<http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
<http://alb.ingress.kubernetes.io/tags|alb.ingress.kubernetes.io/tags>: service_instance=production
<http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
<http://meta.helm.sh/release-name|meta.helm.sh/release-name>: flyte
<http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: flyte
<http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
creationTimestamp: "2022-12-16T17:58:03Z"
finalizers:
- group.ingress.k8s.aws/flyte
generation: 1
labels:
<http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
name: flyte-core
namespace: flyte
resourceVersion: "408320"
uid: 3d84d59a-08fe-41d9-bea3-305413ddd483
spec:
rules:
- http:
paths:
- backend:
service:
name: ssl-redirect
port:
name: use-annotation
path: /*
pathType: ImplementationSpecific
- backend:
service:
name: flyteconsole
port:
number: 80
path: /console
pathType: ImplementationSpecific
- backend:
service:
name: flyteconsole
port:
number: 80
path: /console/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /api
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /api/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /healthcheck
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /v1/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /.well-known
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /.well-known/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /login
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /login/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /logout
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /logout/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /callback
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /callback/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /me
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /config
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /config/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /oauth2
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 80
path: /oauth2/*
pathType: ImplementationSpecific
status:
loadBalancer: {}
- apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
annotations:
<http://alb.ingress.kubernetes.io/actions.ssl-redirect|alb.ingress.kubernetes.io/actions.ssl-redirect>: '{"Type": "redirect", "RedirectConfig":
{ "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
<http://alb.ingress.kubernetes.io/backend-protocol-version|alb.ingress.kubernetes.io/backend-protocol-version>: GRPC
<http://alb.ingress.kubernetes.io/certificate-arn|alb.ingress.kubernetes.io/certificate-arn>: arn:aws:acm:us-west-2:*************:certificate/6c75c8f4-04a1-4aa7-81fa-59c7241e52ba
<http://alb.ingress.kubernetes.io/group.name|alb.ingress.kubernetes.io/group.name>: flyte
<http://alb.ingress.kubernetes.io/listen-ports|alb.ingress.kubernetes.io/listen-ports>: '[{"HTTP": 80}, {"HTTPS":443}]'
<http://alb.ingress.kubernetes.io/scheme|alb.ingress.kubernetes.io/scheme>: internet-facing
<http://alb.ingress.kubernetes.io/tags|alb.ingress.kubernetes.io/tags>: service_instance=production
<http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: alb
<http://meta.helm.sh/release-name|meta.helm.sh/release-name>: flyte
<http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: flyte
<http://nginx.ingress.kubernetes.io/app-root|nginx.ingress.kubernetes.io/app-root>: /console
<http://nginx.ingress.kubernetes.io/backend-protocol|nginx.ingress.kubernetes.io/backend-protocol>: GRPC
creationTimestamp: "2022-12-16T17:58:03Z"
finalizers:
- group.ingress.k8s.aws/flyte
generation: 1
labels:
<http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
name: flyte-core-grpc
namespace: flyte
resourceVersion: "408218"
uid: 114b0b7e-ede6-43f1-8baa-7871600b81c5
spec:
rules:
- http:
paths:
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.AdminService
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.AdminService/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.DataProxyService
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.DataProxyService/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.AuthMetadataService
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.AuthMetadataService/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.IdentityService
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /flyteidl.service.IdentityService/*
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /grpc.health.v1.Health
pathType: ImplementationSpecific
- backend:
service:
name: flyteadmin
port:
number: 81
path: /grpc.health.v1.Health/*
pathType: ImplementationSpecific
status:
loadBalancer: {}
kind: List
metadata:
resourceVersion: ""
selfLink: ""
faint-egg-81361
12/16/2022, 9:17 PMfaint-egg-81361
12/16/2022, 10:54 PMfaint-egg-81361
12/16/2022, 10:54 PM<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: ip
to the annotations created the ALB correctlythankful-minister-83577
faint-egg-81361
12/16/2022, 10:55 PMthankful-minister-83577
thankful-minister-83577
faint-egg-81361
12/16/2022, 10:57 PMfaint-egg-81361
12/16/2022, 10:57 PMfreezing-airport-6809
microscopic-school-54375
02/09/2023, 9:30 AM<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: ip
to the eks-values.yaml
.
That guide is really lovely and much more helpful than the deployment paths on the latest website. I also found that the guide was - apart from that flag above - in great shape still. Any thoughts on adding it to the current version of the website?freezing-airport-6809
freezing-airport-6809
average-finland-92144
02/13/2023, 9:30 PMbroad-monitor-993
02/13/2023, 11:02 PMthe old guide we felt was overly verbose for most people.yeah, on the other hand it sounds like it helped people get setup… based on @microscopic-school-54375’s feedback it may be worth adding at least a link to those manual setup docs as a stop-gap. For context, Ferdinand, maintaining those guides manually was very challenging… however, if we do put up a link to those legacy guides, would you be able to help us figure out what’s out-dated? We are working on a reference implementation for AWS and GCP via terraform, which will nullify the need for the manual docs setup and will simplify how you can stand up a Flyte cluster on the cloud.
microscopic-school-54375
02/14/2023, 9:34 AM<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: ip
to the eks-values.yaml
. Everything else was still on point.freezing-airport-6809
microscopic-school-54375
02/14/2023, 3:40 PMfreezing-airport-6809
freezing-airport-6809
average-finland-92144
02/14/2023, 4:43 PMmicroscopic-school-54375
02/14/2023, 7:17 PMfreezing-airport-6809
hallowed-doctor-67759
02/15/2023, 7:57 PMflyteadmin.service.annotations
and flyteconsole.service.annotations
. But apparently it’ll just work as an annotation on the ingress?average-finland-92144
02/15/2023, 8:30 PM<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>
annotation can be applied both at the Service and Ingress level, with a higher precedence for the Service level. In this case, the annotation for Ingress will be applied unless there's another at the Service level but I guess due to the Flyte architecture and how Ingress is the main entrance point from flytekit, it makes sense to just annotate the Ingress resourcehallowed-doctor-67759
02/15/2023, 8:34 PM