https://flyte.org logo
#ask-the-community
Title
# ask-the-community
g

Guy Harel

02/13/2024, 2:09 PM
Hi all, Are prometheus metrics configured by default when deploying using the flyte-binary chart? It's my understanding that the metrics are exposed when the serviceMonitor is enabled for flyteadmin, however that configuration option is only found in the flyte-core chart and not flyte-binary.We deployed flyte using the flyte-binary chart, and prometheus using the prometheus-community/kube-prometheus-stack chart. I don't see that the flyte-binary chart has configured any ServiceMonitors..
d

David Espejo (he/him)

02/13/2024, 4:41 PM
I see all the necessary pieces for monitoring available on
flyte-core
but not in binary
You'd have to add the extra metrics port using the
extraPorts
in binary (see)
and then create an SM mimicking the config from the template in core But this is me guessing, I haven't tried it
g

Guy Harel

02/13/2024, 4:45 PM
Thanks @David Espejo (he/him) , you referenced the service yaml, does the deployment need that port as well?
BTW : Is there a downside to switching from flyte-binary to flyte-core?
d

David Espejo (he/him)

02/13/2024, 4:47 PM
I see in flyte-core that the flyteadmin K8s service exposes, among other ports, the
http-metrics
- `10254`that then the SM is configured to scrape metrics from
Is there a downside to switching from flyte-binary to flyte-core?
I think it depends on where's your Flyte instance deployed. In EKS for example, it would end up meaning a new IP backing the LB so a change in the DNS entry is needed. From the Flyte POV, the DB schemas are the same, maybe a backup is a good idea but the rest is stateless
g

Guy Harel

02/13/2024, 4:51 PM
We have the ip automated, so that's not an issue.. functionality wise is ir the same code, just running within different pods as opposed to squished together?
d

David Espejo (he/him)

02/13/2024, 4:52 PM
yes, the SDK, API , everything is the same. It's just a different packaging
g

Guy Harel

02/13/2024, 4:53 PM
Got it, thanks! 🙏
I will try the extra ports and SM.
c

Cornelis Boon

04/09/2024, 9:38 AM
@Guy Harel Did this approach work for you in the end? I'm looking into setting up monitoring via prom/grafana for flyte-binary right now. Or did you switch to flyte-core in the end?
g

Guy Harel

04/09/2024, 9:49 AM
Actually I don't get it to work yet but other things took priority..
c

Cornelis Boon

04/09/2024, 9:50 AM
Ah, too bad. Thank you for your answer. The approach does look promising
2 Views