creamy-energy-2403
07/21/2023, 11:05 AMflytectl
problem which I can't solve for now; maybe someone will be able to help me with it?
$ HTTPS_PROXY=localhost:8080 flytectl --admin.insecureSkipVerify -d development -p flyteexamples get tasks
Error: Connection Info: [Endpoint: dns:///flyte.dexterenergyservices.com, InsecureConnection?: false, AuthMode: Pkce]: rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 502 (Bad Gateway); transport: received unexpected content-type "text/html; charset=UTF-8"
I'm using mitmproxy to intercept the flytectl
traffic (hence the HTTPS_PROXY
env var and --admin.insecureSkipVerify
additions) and I can see that flytectl
issues a request to /flyteidl.service.AdminService/ListTasks
endpoint.
It results in HTTP 502 response from the GCP Load balancer claiming that "server encountered a temporary error".
This is where I am stuck for now - I do have flyteadmin
Service running and the corresponding flyteadmin
Pods also green and pass the healthchecks, but I can't figure out how to confirm if flyteadmin
is actually running and responding.
Best thing I've found so far is that if I request the /me
endpoint (which is sent to the HTTP interface of the flyteadmin:80
service), it responds with the following response:
{
"error": "unknown service flyteidl.service.IdentityService",
"code": 12,
"message": "unknown service flyteidl.service.IdentityService"
}
And that looks suspicious :) Maybe it's a well-known problem? I would appreciate any help with itcreamy-energy-2403
07/21/2023, 11:20 AMIngress
instead of nginx or contour
* separateGrpcIngress
is set to false
(maybe that's the reason?)
* flyteadmin
Service has this cloud.google.com/app-protocols={"grpc":"HTTP2"}
annotation
* I can open Flyte console in the browser.
I'll try to poke the separateGrpcIngress
setting in a meanwhile