Is it possible to expose a prometheus collection endpoint in flyte-binary deployment?

I think the metrics are exposed by default on

/metrics

for the single binary (see thread)

Yup, I had to add:

propeller:
      prof-port: 10254
      metrics-prefix: "flyte:"
    scheduler:
      profilerPort: 10254
      metricsScope: "flyte:"
    flyteadmin:
      profilerPort: 10254
      metricsScope: "flyte:"

under

configuration.inline

in my helm values. and

service:
  extraPorts:
  - name: http-metrics
    protocol: TCP
    port: 10254

on the top level of helm values. And then could configure a serviceMonitor for flyte like this:

apiVersion: <http://monitoring.coreos.com/v1|monitoring.coreos.com/v1>
kind: ServiceMonitor
metadata:
  name: flytemonitoring
  namespace: flyte
spec:
  selector:
    matchLabels:
      <http://app.kubernetes.io/instance|app.kubernetes.io/instance>: flyte-backend
  endpoints:
  - port: http-metrics
    path: /metrics

@gentle-tomato-480 thank you so much!! I figured out the extraPorts and servicemonitor Why do I need the first part? any metrics that's missing if it's not added? also, under which values config I need to add this part?

It's been a while since I set it up, but I believe the first part I had found in core values chart: https://github.com/flyteorg/flyte/blob/5cc7f583d05b27efb13d799177551be17f7bc654/charts/flyte-core/values.yaml#L692-L695 https://github.com/flyteorg/flyte/blob/5cc7f583d05b27efb13d799177551be17f7bc654/charts/flyte-core/values.yaml#L717-L720 https://github.com/flyteorg/flyte/blob/5cc7f583d05b27efb13d799177551be17f7bc654/charts/flyte-core/values.yaml#L827-L828 The top one you should put under

configuration.inline

in your

values.yaml

e.g.

configuration:
  inline:
    propeller:
      prof-port: 10254
      metrics-prefix: "flyte:"
    scheduler:
      profilerPort: 10254
      metricsScope: "flyte:"
    flyteadmin:
      profilerPort: 10254
      metricsScope: "flyte:"

I'm not sure if it's necessary, but it feels complete.

Thanks a lot @gentle-tomato-480