allen
08/08/2022, 2:45 PMflyteconsole
due to the following error in the AWS ELB Controller logs:
{
"level": "error",
"ts": 1659712067.8667307,
"logger": "controller",
"msg": "Reconciler error",
"controller": "ingress",
"name": "flyte",
"namespace": "",
"error": "InvalidParameter: 1 validation error(s) found.\n- minimum field value of 1, CreateTargetGroupInput.Port.\n"
}
Seems like the issue is https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/1695 due to the service that the ingress routes to having type clusterip
instead of nodeport
, yet the helm chart for flyteconsole define the service as type clusterip
. So I’m wondering if there’s something else that I’m not configuring correctly.
If anyone else has run into something similar help would be very welcome! ThanksJacob Wang
08/08/2022, 3:10 PM<http://alb.ingress.kubernetes.io/target-type|alb.ingress.kubernetes.io/target-type>: 'ip'
and it works, the address shows upallen
08/08/2022, 3:27 PMJacob Wang
08/08/2022, 3:28 PMelb adresss/console
, but when I execute flytectl create project
, it returns:
{
"json": {
"src": "main.go:13"
},
"level": "error",
"msg": "rpc error: code = Unavailable desc = connection closed",
"ts": "2022-08-08T17:26:10+02:00"
}
Ketan (kumare3)
allen
08/08/2022, 3:46 PMKetan (kumare3)
Jacob Wang
08/08/2022, 3:50 PMKetan (kumare3)
allen
08/08/2022, 4:38 PMflytectl create
I get Error: rpc error: code = Unavailable desc = connection closed
. I tried changing HTTP2 to GRPC for the grpc ingress: <http://alb.ingress.kubernetes.io/backend-protocol-version|alb.ingress.kubernetes.io/backend-protocol-version>: GRPC
but it didn’t work. See: https://github.com/flyteorg/flyte/pull/2702/files#diff-33b4463f6057591a533425d1f947752711a81da1952ff745ed9fae049e155995
@Ketan (kumare3) happy to update docs once I get this up and running!Jacob Wang
08/08/2022, 4:40 PMallen
08/08/2022, 8:05 PM<http://alb.ingress.kubernetes.io/backend-protocol-version|alb.ingress.kubernetes.io/backend-protocol-version>: HTTP1
and disabling albSSLRedirect: false
in the config. Now when I try to access flytectl, I get Unavailable desc = connection error: desc = "transport: authentication handshake failed: x509: "<cdn for certificate domain>" certificate name does not match input
. Trying to make inroads using this workaroundKetan (kumare3)
Yuvraj
08/09/2022, 5:33 AMflytectl get projects --admin.insecureSkipVerify --logger.level=10
Jacob Wang
08/09/2022, 7:33 AM{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2022-08-09T09:31:24+02:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [root] updated. No update handler registered.","ts":"2022-08-09T09:31:24+02:00"}
{"json":{"src":"viper.go:400"},"level":"debug","msg":"Config section [admin] updated. Firing updated event.","ts":"2022-08-09T09:31:24+02:00"}
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [files] updated. No update handler registered.","ts":"2022-08-09T09:31:24+02:00"}
{"json":{"src":"client.go:183"},"level":"error","msg":"failed to initialize token source provider. Err: failed to fetch auth metadata. Error: rpc error: code = Unavailable desc = connection closed","ts":"2022-08-09T09:31:25+02:00"}
{"json":{"src":"client.go:188"},"level":"warning","msg":"Starting an unauthenticated client because: can't create authenticated channel without a TokenSourceProvider","ts":"2022-08-09T09:31:25+02:00"}
{"json":{"src":"client.go:64"},"level":"info","msg":"Initialized Admin client","ts":"2022-08-09T09:31:25+02:00"}
Error: rpc error: code = Unavailable desc = connection closed
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unavailable desc = connection closed","ts":"2022-08-09T09:31:25+02:00"}
Yuvraj
08/09/2022, 7:34 AMJacob Wang
08/09/2022, 7:34 AMaffinity
section for each pods if you have pending state pods, you can ignore this if you don’t have pending state pods because of affinity
3. In the Connecting to Flyte Section if you are using a self-signed certificates, in the ~/.flyte/config.yaml
, write insecure: false
instead of true
since you have a certificate anyways so it’s not insecure. Leave insecureSkipVerify: true
then your certificate won’t be verified. (Should not apply to production as mentioned in setup)Yuvraj
08/09/2022, 8:37 AMkubectl get pod -n flyte
Ketan (kumare3)
Jacob Wang
08/09/2022, 2:08 PMallen
08/09/2022, 2:34 PM{"json":{"src":"client.go:64"},"level":"info","msg":"Initialized Admin client","ts":"2022-08-09T10:33:18-04:00"}
Error: rpc error: code = Unknown desc = : HTTP status code 464; transport: missing content-type field
{"json":{"src":"main.go:13"},"level":"error","msg":"rpc error: code = Unknown desc = : HTTP status code 464; transport: missing content-type field","ts":"2022-08-09T10:33:18-04:00"}
I’m using ALB. I have <http://alb.ingress.kubernetes.io/backend-protocol-version|alb.ingress.kubernetes.io/backend-protocol-version>: GRPC
and have tried the config settings that @Jacob Wang suggested above
I’m running: flytectl get projects --admin.insecureSkipVerify --logger.level=10
Yuvraj
08/09/2022, 3:25 PM