Hi all; I’m working on setting up an on prem instance of flyte but keep running into an issue with the liveness and readiness probes:

Readiness probe failed: Get “http://<pod IP>:8088/healthcheck”: dial tcp <pod IP>:8088: connect: connection refused

Same error for the liveness probe. Has anyone ran into this before or does anyone have an idea of how to solve it? Additionally, if I comment out the liveness and readiness probes, the installation will succeed, but when I try to access the 8088 port on the service or pod I get a ‘connection refused’ response. Any ideas

What chart are you using?

flyte-binary

what pod are you testing? can you hit the service instead? can you exec into the binary pod and install curl and then curl localhost 8088?

I did try doing that but on another pod from another service but in the same namespace — from there I tried curling the 8088 port of the service like you described but still got ‘connection refused’

@Brennan Tolman If you have kubectl access, you could try using

kubectl logs deployment/<flyte-binary deployment name> -n <namespace of your flyte deployment>

And then see what the logs say. Whenever I run into an error message like that, it usually means some part of the service is failing due to a misconfiguration

@Tommy Nam Thanks for the response. I commented out the liveness and readiness probe and checked the logs — I see a bunch of successful Postgres commands and no errors. Same thing happens when I uncomment the probes. Is there a specific thing you are referring to when you say you often run into things like this with misconfiguration?