https://flyte.org logo
Powered by
s

Sonja Ericsson

07/06/2022, 1:37 PM
hey! is there a metric in propeller to measure failures from plugins? We use these recording rules currently, but each time we add a new plugin we need to go and add it to the rules 🧵
Copy code
- record: computed:propeller:all:node:plugin:failure_unlabeled
  expr: |    label_replace(flyte:propeller:all:node:plugin:bigquery_ddl_create_dataset_task_bigquery_api_failure_unlabeled, "plugin", "bigquery_ddl_create_dataset_task_bigquery_api", "", "")

- record: computed:propeller:all:node:plugin:failure_unlabeled
  expr: |
    label_replace(flyte:propeller:all:node:plugin:flink_flink_failure_unlabeled, "plugin", "flink_flink", "", "")
k

katrina

07/06/2022, 5:18 PM
hey @Sonja Ericsson what do you mean by failures from plugins?
s

Sonja Ericsson

07/06/2022, 5:20 PM
I guess if a reconcile failure is due to any specific plugin having problems
k

katrina

07/06/2022, 5:23 PM
this is where we do the plugin to task type resolution but it looks like we only log if we encounter issues. are you mostly trying to make sure if you configure a new plugin it will actually be used for a specific task type?
s

Sonja Ericsson

07/07/2022, 7:02 AM
We would like to understand if problems in propeller or if a set of workflows are not progressing is due to a specific plugin. For example when handle is called on a specific plugin, if some exception comes from the part of the code of the plugin.
k

katrina

07/07/2022, 3:56 PM
Are the existing error logs sufficient in this case?
s

Sonja Ericsson

07/07/2022, 3:58 PM
yes it works! it would be awesome though if it was possible. we had some issues recently where all our metrics looked fine, but one plugin was misbehaving due to some change in the general propeller config, so it would have been nice to be able to alert on it
k

katrina

07/07/2022, 4:17 PM
gotcha, please feel free to open a PR!
👍 1
2 Views
Powered by